Virtual workspace viewport following in collaboration systems

ABSTRACT

A collaboration system is provided and includes memory storing collaboration data including a spatial event map locating events in a virtual workspace and a local display including a screen space. The collaboration system further includes logic to identify a local client viewport in the virtual workspace, logic to render, in the screen space on the local display, graphical objects identified by events in the spatial event map that have locations within the local client viewport, logic to change a location and a dimension of the local client viewport in the virtual workspace in response to input signals, logic to render, in the screen space on the local display, graphical objects identified by events in the spatial event map that have locations within the changed local client viewport, and logic to generate a viewport change event in response to the change of the local client viewport in response to input signals.

RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional applicationSer. No. 17/684,354, entitled “Virtual Workspace Viewport Following inCollaboration Systems”, filed 1 Mar. 2022 (Attorney Docket No. HAWT1019-11), which is a continuation of U.S. Non-Provisional applicationSer. No. 17/027,571, entitled “Virtual Workspace Viewport Following inCollaboration Systems”, filed 21 Sep. 2020 (Attorney Docket No. HAWT1019-10), which is a continuation of U.S. Non-Provisional applicationSer. No. 15/147,576, entitled “Virtual Workspace Viewport Following inCollaboration Systems”, filed 5 May 2016 (Attorney Docket No. HAWT1019-2A), which claims the benefit of U.S. Provisional Application No.62/157,911, entitled “System and Method for Emulation of a ViewportWithin a Shared Workspace”, filed 6 May 2015 (Attorney Docket No. HAWT1019-1). The above-referenced applications are incorporated herein byreference.

The following, commonly owned, U.S. patent application is incorporatedby reference as if fully set forth herein, U.S. application Ser. No.14/090,830, entitled “Collaboration System Including A Spatial EventMap”, filed 26 Nov. 2013 (Attorney Docket No. HAWT 1011-2).

FIELD OF THE TECHNOLOGY DISCLOSED

The technology disclosed relates to apparatuses, methods, and systemsfor digital collaboration, and more particularly to digital displaysystems which facilitate multiple simultaneous users having access toglobal workspace data.

DESCRIPTION OF RELATED ART

Digital displays are often used for interactive presentations and otherpurposes in a manner analogous to whiteboards. Some displays arenetworked and can be used for collaboration, so that modifications madeto the display image on one display are replicated on another display.Collaboration systems can be configured to operate collaborationsessions in which users located at different client platforms share aworkspace as described in our co-pending U.S. application Ser. No.14/090,830, entitled “Collaboration System Including A Spatial EventMap”, filed 26 Nov. 2013 (US 2014-0222916-A1, published 7 Aug. 2014).The distributed nature of such systems allows multiple users indifferent places to interact with, and change, data in the sameworkspace at the same time, and also at times when no other user isobserving the workspace. Also, the workspace can be very large,essentially unbounded in some systems.

One problem associated with collaboration systems using largeworkspaces, relates to navigation around the workspace. Because theworkspace can be essentially unbounded, and users can place graphicalobjects anywhere in the workspace, it can be difficult to discover andtrack the work being done by collaborators.

SUMMARY

A system is disclosed that supports the storing and tracking of aplurality of collaboration sessions, each of which is accessible acrossmultiple devices and locations. The technology disclosed includes amethod for one client to find and track on their display thetransactions generated by another client within a shared workspace.

One system described herein comprises one or more data processorsincluding memory storing computer programs for a database including oneor more workspace data structures for corresponding workspaces. Aworkspace data structure can include a spatial event map for a specificworkspace.

The system described includes a first network node including a displayhaving a physical display space, a user input device, a processor and acommunication port. The first network node can be configured with logicto establish communication with one or more other network nodes, whichcan include for example server-side network nodes and peer client-sidenetwork nodes, as a participant client in a workspace session. The firstnetwork node can have memory, or have access to memory, to storecollaboration data identifying graphical targets having locations in avirtual workspace used in the workspace session. The collaboration datacan be a spatial event map, as described herein, or other type of datastructure, including locations in a virtual workspace of respectivegraphical targets. The first network node in this system has logic todefine a local client viewport having a location and dimensions withinthe workspace, and to map the local client viewport to a local clientscreen space in the physical display space at the first network node.The first network node can also provide a user interface displaying alist of participant clients in the session at other network nodes, andfor receiving input indicating a selected other participant client fromthe list. The first network node can receive messages containing alocation in the workspace of a participant client viewport in use at theselected other participant client. Using the location of the participantclient viewport, the first network nodes can update the location of thelocal client viewport to the identified location of the participantclient viewport in use at the selected other participant client, andrender on the screen space graphical targets having locations within theupdated local client viewport. This implements an optional behavior fora client in the collaboration session than can be called “follow.”

A node for use in a collaboration system is described that comprises adisplay having a physical display space, a user input device, aprocessor and a communication port, the processor being configured withlogic to implement the follow mode. The logic can be configured to:

-   -   establish communication with one or more other network nodes;    -   store collaboration data identifying graphical targets having        locations in a virtual workspace, the collaboration data        including locations in the virtual workspace of the identified        graphical targets;    -   compute a mapping of a local client viewport having a location        and dimensions in the virtual workspace to a local client screen        space having dimensions in the physical display space, and        render graphical targets having locations in the local client        viewport to the local client screen space as a function of the        mapping; and    -   determine the location and dimensions in the virtual workspace        of the local client viewport, the logic to determine including    -   (i) a first mode under local control such as touch screen and        mouse gestures for panning and zooming to change the local        client viewport in response to user input at the first network        node; and    -   (ii) a second mode, or follow mode, to monitor messages        containing locations and changes of location in the virtual        workspace of a remote client viewport from a second network node        in the one or more network nodes, and to change the local client        viewport using the locations and changes of location contained        in the messages.

The local client screen space in a network node has an aspect ratio, anda resolution including a number of pixels determined by the display anddisplay driver at the network node. The resolution can be static in thesense that it is unchanging during changes of the local client viewportin the follow mode. The messages containing locations and changes oflocation include a specification of dimensions having an aspect ratio ofthe remote client viewport. The logic to change the local clientviewport can define dimensions of the local client viewport as afunction of differences between the aspect ratio of the local clientscreen space and the aspect ratio of the remote client viewport. In thisway, the graphical objects rendered on remote client screen space usingthe remote client viewport are reproduced on the local client screenspace using the local client viewport without clipping.

Also, the local client screen space has a screens space resolution, thatcan be static, including a number of pixels, and the dimensions of thelocal client viewport define a changeable resolution including a numberof virtual pixels in the virtual workspace. The logic to compute amapping determines a zoom factor based on differences in the static,screen space resolution and the changeable resolution.

A node for use in a collaboration system is described that comprises adisplay having a physical display space, a user input device, aprocessor and a communication port, the processor being configured withlogic to implement dynamic location marker creation, movement, searchingand selection. The logic at a particular network can be configured to:

-   -   establish communication with one or more other network nodes;    -   store collaboration data identifying graphical targets having        locations in a virtual workspace, the collaboration data        including locations in the virtual workspace of the identified        graphical targets;    -   compute a mapping of a local client viewport having a location        and dimensions in the virtual workspace to a local client screen        space having dimensions in the physical display space, and        render graphical targets having locations in the local client        viewport to the local client screen space as a function of the        mapping;    -   provide a user interface displaying a list of location markers        in the workspace, a location marker having a marked location in        the workspace, and for receiving input indicating a selected        location marker from the list;    -   determine the location and dimensions in the virtual workspace        of the local client viewport, including to update the location        of the local client viewport to the marked location of the        selected location marker; and    -   render on the screen space graphical targets having locations        within the local client viewport.

The logic to determine the location and dimensions of the local clientviewport includes logic to change the local client viewport in responseto pointer or touch based user input gestures at the first network nodeindicating movement or zoom of the local client viewport.

The user interface displaying a list of location markers also displays aselectable entry for default location for the workspace and includinglogic to update the location of the local client viewport to the defaultlocation upon selection of the selectable entry.

The logic can be configured, so that in response to the selection of theselected location marker, the local client viewport location is changedwithout changing its dimensions in the virtual workspace.

The user interface displaying a list of location markers includes asearch function based on querying location marker tags in embodimentsdescribed herein.

The actions usable in the records of an events in the log of eventsinclude creation, movement and deletion of location markers.

The network nodes can include logic to send messages to other networknodes, the messages identifying events including creation or movement ofa location marker, and the logic is responsive to receipt of a messagefrom a second network node to update the list of location markers.

In a described implementation, the first network node can accept inputdata from the user input device creating events relating to modificationof the local client viewport, and create a client viewport data objectdefining the location of the local client viewport within the workspace.The first network node can send the client viewport data object to oneor more other network nodes participating in the session, to support thefollow option.

Also described is a system comprising a first network node providing auser interface displaying a list of location markers in the workspace inthe session, where a location marker has a marked location in theworkspace. The user interface can receive input indicating a selectedlocation marker from the list. In response to selection of a selectedlocation marker, the system can update the location of the local clientviewport to the marked location and render on the screen space graphicaltargets having locations within the updated local client viewport.

In another implementation, a system can include a spatial event mapsystem, comprising a data processor, and memory storing a spatial eventmap which locates events in a virtual workspace, and providescommunication resources for collaborative displays.

Also, methods of operating client nodes in a collaboration session asdescribed herein, and non-transitory computer readable media storingcomputer programs, implement logical functions as described herein, fora client in a collaboration session are described.

The above summary is provided in order to provide a basic understandingof some aspects of the collaboration system described herein. Thissummary is not intended to identify key or critical elements of thetechnology disclosed or to delineate a scope of the technologydisclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

The included drawings are for illustrative purposes and serve to provideexamples of structures and process operations for one or moreimplementations of this disclosure. These drawings in no way limit anychanges in form and detail that can be made by one skilled in the artwithout departing from the spirit and scope of this disclosure. A morecomplete understanding of the subject matter can be derived by referringto the detailed description and claims when considered in conjunctionwith the following figures, wherein like reference numbers refer tosimilar elements throughout the figures. The technology disclosed willbe described with respect to specific embodiments thereof, and referencewill be made to the drawings, which are not drawn to scale, and inwhich:

FIGS. 1A, 1B, and 1C (collectively FIG. 1 ) illustrate oneimplementation of a system of network nodes that collaborate within avirtual workspace.

FIG. 2 illustrates one implementation of two network nodes havingviewports in distinct areas of an unbounded virtual workspace.

FIG. 3 illustrates how a viewport of one network node can be changedwithout affecting the viewports of other network nodes.

FIGS. 4A, 4B, and 4C (collectively FIG. 4 ) illustrate a first networknode “following” a second network node.

FIG. 5 is a flowchart that illustrates one implementation of a firstnetwork node following a second network node.

FIG. 6 illustrates a message map between a first client-side networknode, a server-side network node, and a second client-side network nodefor the first client-side network node to follow the second client-sidenetwork node.

FIGS. 7A, 7B, and 7C (collectively FIG. 7 ) illustrate examples of alocation marker.

FIGS. 8A and 8B (collectively FIG. 8 ) illustrate selecting a locationmarker from a menu.

FIGS. 9A, 9B, and 9C (collectively FIG. 9 ) illustrate an example ofadding a location marker to a virtual workspace.

FIG. 10 is a flowchart that illustrates the selection of a locationmarker from a menu.

FIG. 11 illustrates a message map for communication of location markersbetween network nodes.

FIG. 12 illustrates example aspects of a digital display collaborationenvironment.

FIG. 13 illustrates additional example aspects of a digital displaycollaboration environment.

FIGS. 14A, 14B, 14C, 14D, 14E, and 14F (collectively FIG. 14 ) representdata structures which can be part of workspace data maintained by adatabase at the collaboration server 1305.

FIG. 15 is a simplified block diagram of a computer system, or networknode, which can be used to implement the client-side functions or theserver-side functions in a distributed collaboration system.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the technology disclosed and is provided in thecontext of particular applications and requirements. Variousmodifications to the disclosed embodiments will be readily apparent tothose skilled in the art, and the general principles defined herein maybe applied to other embodiments and applications without departing fromthe spirit and scope of the technology disclosed. Thus, the technologydisclosed is not intended to be limited to the embodiments shown but isto be accorded the widest scope consistent with the principles andfeatures disclosed herein.

The “unlimited workspace” problem includes the need to track how peopleand devices interact with the workspace over time. In oneimplementation, this can be addressed by allowing a first system tofollow the actions of a second system. In another implementation, afirst system can save a location with a location marker and make thelocation marker available to a second system. In order to solve thiscore problem, a Spatial Event Map has been created, which includes asystem architecture supporting collaboration using a plurality ofspatial event maps and a plurality of collaboration groups. The SpatialEvent Map contains information needed to define objects and events in aworkspace. The Spatial Event Map comprises a data structures specifyingevents having locations in a virtual collaboration space. The events,maps of events in the space, and access to the space by multiple users,including multiple simultaneous users support collaboration from usersaround the globe.

Space: In order to support an unlimited amount of spatial informationfor a given collaboration session, a way is provided to organize avirtual space termed the workspace, which can for example becharacterized by a two-dimensional Cartesian plane with essentiallyunlimited extent in one or both of the dimensions for example, in such away that new content can be added to the space, that content can bearranged and rearranged in the space, that a user can navigate from onepart of the space to another, and that a user can easily find neededthings in the space when required.

Events: Interactions with the workspace are handled as events. People,via tangible user interface devices, and systems can interact with theworkspace. Events have data that can define or point to a targetgraphical object to be displayed on a physical display, and an action ascreation, modification, movement within the workspace and deletion of atarget graphical object, and metadata associated with them. Metadata caninclude information such as originator, date, time, location in theworkspace, event type, security information, and other metadata.

Tracking events in a workspace enables the system to not only presentthe spatial events in a workspace in its current state, but to share itwith multiple users on multiple displays, to share relevant externalinformation that may pertain to the content, and understand how thespatial data evolves over time. Also, the spatial event map can have areasonable size in terms of the amount of data needed, while alsodefining an unbounded workspace.

There can be several different kinds of events in the system. Events canbe classified as persistent events, also referred to as history eventsthat are stored permanently, or for a length of time required by thesystem for maintaining a workspace during its useful life. Events can beclassified as ephemeral events that are useful or of interest for only ashort time and shared live among other clients involved in the session.Persistent events may include history events stored in an undo/playbackevent stream, which event stream can be the same as or derived from thespatial event map of a session. Ephemeral events may include events notstored in an undo/playback event stream for the system. A spatial eventmap, or maps, can be used by a collaboration system to track the timesand locations in the workspace in some embodiments of both persistentand ephemeral events on workspaces in the system.

Map: A map of events in the workspace can include the sum total ofdiscrete spatial events that relate to graphical objects havinglocations in the workspace. Events in the map of a workspace can be“mapped” to a physical display or screen that has a displayable areareferred to herein as a screen space, of specific size. A client canspecify a viewport in the workspace, having a location and a dimensionin the workspace, and then map the events in the viewport area to thescreen space for display.

Multi-User Access: One key characteristic is that all users, or multipleusers, who are working on a workspace simultaneously, should be able tosee the interactions of the other users in a near-real-time way. Thespatial event map allows users having displays at different physicallocations to experience near-real-time events, including both persistentand ephemeral events, within their respective viewports, for all userson any given workspace. The collaboration system architectures describedherein enable operation of many workspaces and many user groups.

Widget: A widget is a component of a workspace that the user caninteract with or view, e.g. Notes, Images, Clocks, Web Browsers, VideoPlayers, Location Markers, etc. A Window is a widget that is arectangular region with two diagonally opposite corners. Most widgetsare also windows.

In a collaborative environment, it can be beneficial to see what othersare working on within the environment. The technology disclosed allows afirst network node to follow the events that occur on a second networknode without any significant increase in network utilization. This canbe accomplished by exchanging messages carrying simple text eventrecords that can include JSON data structures or the like, rather thansending images between network nodes. The first network node receivesdescriptions of events from all other participating network nodes withina virtual workspace, and stores at least some of them in a local log.The first network node also creates its own events, and stores at leastsome of them in the local log. The first network node has a viewportinto the virtual workspace that can include any number of graphicaltargets defined by the events. The first network node can render theobjects described by the event records that have coordinates within itsviewport, ignoring the event records describing events relating tographical object located outside of its viewport.

In one example, an operator of the first network node might beinterested in watching the events that are happening within a viewportof the second network node as they occur, and as the viewport of thesecond network node is moved around in the workspace. The first networknode can extract event information from the local log that describes theviewport of the second network node, and the graphical targets withinthe viewport, of the second network node, and render those graphicaltargets on a local screen space. In another example, an operator of thefirst network node might want to visit a place within the workspace thatwas saved by the second network node as a location marker. The firstnetwork node can extract event information from the local log thatdescribes the workspace, to identify a location marker saved by thesecond network node, and then move its viewport to the location of thelocation marker.

An environment is illustrated by FIGS. 1A-1C, describing a virtualworkspace with graphical targets, a viewport, and a screen space. FIG.1A illustrates a first network node 101 with a physical display space103 that includes a screen space 105 allocated for use in thecollaboration session. As illustrated, the screen space 105 can becoextensive with the physical display space 103 at the first networknode, or can consist of an allocated portion, like a window, in thephysical display space. The physical display space has a resolution interms of a number of pixels and an aspect ratio set by the local networknode that is independent of the workspace. The resolution in theworkspace is based on the coordinate system used and can include anessentially unbounded number of virtual points that can identifylocations in the virtual workspace. In one example, the screen space 105includes the entire displayable area of, and has the same resolution as,the physical display space 103. In another example, the screen space canbe smaller than the physical display space.

In the illustrated example, the network node can include touch sensorson the screen space 105 that can perform as a user input device. Thecollaboration system client on the network node can access a local logfile 111 that can store event records defining spatial event map orother type of data structure representing contents of a currently usedworkspace. In this example, a set of graphical targets 191, and alocation marker 195 are displayed in the screen space 105.

FIG. 1B illustrates a virtual workspace 165, a first viewport 175, and asecond viewport 177 that have locations and dimensions in the workspace.A plurality of graphical targets 191 and a graphic target of type firstlocation marker 195, have locations within the first viewport 175.Graphical targets 193 have locations within the second viewport 177. Asecond location marker 197 has a location within the virtual workspace165 outside the boundaries of both the first viewport 175 and the secondviewport 177.

FIG. 1C shows a second network node 151 with a physical display space153 and a screen space 155. In the illustrated example, the network nodecan include touch sensors that can perform as a user input device. Thenode can have access to a local log file 161 that can store eventrecords defining a spatial event map or other type of data structurerepresenting contents of a virtual workspace, such as the same workspacecurrently in use at the first network node. In this example, theviewport of the second network node is viewport 177, and maps to thescreen space 155. As a result of the mapping, a set of graphical targets193 is displayed in the screen space 155.

A network node can generate an event to record the creation of agraphical target such as a text box, a location marker, a web page, or aviewport within a virtual workspace. The event can include the locationof the graphical target within the virtual workspace, a time of theevent, and a target identifier of the graphical target. The network nodecan then communicate the event to other network nodes participating inthe workspace. Each participating network node can store the event inits local log 111, 161. In this example, an event exists in the locallog 111, 161 for each of the events creating or modifying or moving thegraphical targets 191, 193, the location markers 195, 197, and theviewports 175, 177 within the virtual workspace 165. The graphicaltargets of the events can be rendered on the screen space 105, 155 by aprocessor with logic to render the graphical targets.

The processor includes logic to render graphical targets havinglocations in a viewport to the screen space, and to render only thosegraphical targets, or portions of graphical targets, that are within theboundaries of the viewport, using a zoom level that is a function of thelocal screen space resolution and the dimensions of the local clientviewport.

A screen space can have a fixed aspect ratio, and fixed resolution interms of pixels per line, and lines per screen. This aspect ratio andresolution can be used to compute the mapping of the viewport to thescreen space. For example, a starting viewport in the workspace caninclude an array of 1000 points by 1000 lines. The screen space can havethe same resolution of 1000 by 1000. However, if a user executes a zoomout operation, the screen space resolution remains the same, but theworkspace resolution increases to for example 2000 points by 2000 lines.In this case, the graphical targets of the events in the larger viewportare scaled to fit within the smaller number of pixels in the screenspace as a function of the zoom factor. Likewise, if the user executes azoom in operation, the screen space resolution remains the same, but theworkspace resolution decrease to for example 500 points by 500 lines. Inthis case, the graphical targets of the events in the smaller viewportare scaled to fit within the larger number of pixels in the screenspace. A viewport can be specified by a location in the workspace, anaspect ratio of the client screen space, and a zoom level, or ratio ofresolution of the viewport compared to that of the screen space.

This allows various devices such as mobile devices, computers, and wallsto display respective viewports at a common zoom level and at aspectratios that match the respective screen spaces. The technology disclosedallows clients to specify viewports independently, so that two viewportsmay overlap. In one example, a first user modifies a viewport so that itincludes an area already included in the viewport of a second user. Inthis example, the viewports are independent of each other, and oneviewport can be modified without affecting the other. In anotherexample, a first user “follows” a second user, whereby the viewport ofthe first user is determined by the viewport specifications of thesecond user. In this case, if the screen spaces have the same aspectratio and resolution, then the screen space of the first user displaysthe same graphical targets and can be like a replica of the screen spaceof the second user. In the case in which the aspect ratio and/orresolutions do not match, then the following node can translate thedimensions of the remote client viewport to a local client viewportbased on the aspect ratio and resolution of the local client screenspace.

FIGS. 2, 3, and 4 illustrate the process of a first network nodefollowing a viewport in use at a second network node.

FIG. 2 illustrates one implementation of two network nodes havingviewports in distinct areas of a virtual workspace. FIG. 2 illustrates avirtual workspace 210 comprising a first viewport 230 and a secondviewport 220. A set of graphical targets 235 having locations in thefirst viewport 230, and a set of graphical targets 225 and a particulargraphical target 215 having locations within the second viewport 220 areshown. FIG. 2 also illustrates schematically a first screen space 250 ata first network node and a second screen space 275 at a second networknode. In this illustration, the first viewport 230 containing the set ofgraphical targets 235 is mapped to the first screen space 250, and theset of graphical targets 235 are rendered on it. The second screen space275 has the graphical targets (225 and 215) in second viewport 220rendered on it. Each viewport contains a different set of graphicaltargets. In another example, some or all of the graphical targets canexist outside of the viewports.

FIG. 3 illustrates how a viewport of one network node can be changedwithout affecting the viewports of other network nodes. FIG. 3illustrates a second network node changes the dimensions in theworkspace of viewport 320 without affecting the viewport 330 of a firstnetwork node. In this example, FIG. 3 is based on FIG. 2 . FIG. 3illustrates a virtual workspace 310 comprising a first viewport 330 anda second viewport 320. The first viewport 330 envelops a set ofgraphical targets 335. The second viewport envelops a set of graphicaltargets 325. A graphical target 315 exists within the virtual workspace310 outside of the area enveloped by the first viewport 330 or thesecond viewport 320. FIG. 3 also illustrates a first screen space 350and a second screen space 375. The first screen space 350 has the firstviewport 330 containing the set of graphical targets 335 rendered on it.The second screen space 375 has the second viewport 320 containing theset of graphical targets 325 rendered on it. The graphical target 315within the virtual workspace 310 exists outside of either of theviewports 330, 320, and is not rendered on either the first screen space350 or the second screen space 375.

A difference between FIG. 2 and FIG. 3 is that the viewport 320 has beenchanged in FIG. 3 to exclude the graphical target 315. A viewport can bechanged by changing the zoom level of the viewport by using, forexample, a variety of input devices and gestures, including, forexample, a zoom wheel on a mouse, or combinations of keys pressed on akeyboard. A viewport can also be changed by a touch screen or mousegesture that includes “grabbing” a location on the screen space with amouse or other pointer, and then moving the mouse.

A display is a device comprised of an array of pixels. Complex displays,such as walls 1202, comprise multiple displays in a rectangular array,and have consecutive pixels in the X and Y coordinates managed by acontroller. In one implementation, a display can have multiple windows,each window comprising a separate screen space.

For example, a workspace can have a set of objects laid out betweencoordinates x0=−10000, y0=4500 and x1=5200, y1=−1400, in abstract units.If a client wants to see that set of objects, then it defines a viewportwith those coordinates, and then renders that viewport within its screenspace, mapping the abstract units of the workspace to the physical unitsof displayable pixels. If the client wants to see more of the workspace,they can zoom out so that more distributed x0, y0, x1, y1 coordinates ofthe viewport map to the available space in the screen space. If theywant to see a smaller area of the workspace, they zoom in to whateverx0′, y0′, x1′, y1′ coordinates they want, and those coordinates aremapped to the screen space of the client. In rendering the viewport tothe screen space, scaling of the contents of the viewport can beaccomplished through standard graphical scaling techniques.

A change to a viewport is characterized as an event which, in thisexample, causes a “vc” (Viewport Change) record to be created, stored inthe local log, and communicated to other participant clients in thecollaborative workspace. An example of a “vc” record for use in an APIlike that described below, is represented as follow:

// server <--> client [sender-id, ″vc″, viewport-rect]  • viewport-rectan array in the form [x1, y1, x2, y2] representing the corners   of asection of the workspace being used as the viewport on the sendingclient.

Other network nodes participating in the collaborative workspace canreceive the “vc” record, where it can be stored in the local log. Theother network nodes will not act on the “vc” record unless they invokelogic to follow the network node that generated the “vc” record.

FIGS. 4A, 4B, and 4C illustrate a first network node “following” theviewport in use at a second network node. FIG. 4A illustrates a virtualworkspace 410 comprising a first viewport 430, a second viewport 420Aand a third viewport 420B. FIG. 4A also illustrates a first screen space450 and a second screen space 475. The first viewport 430 specifies anarea in the virtual workspace 410 that envelops a set of graphicaltargets 435. The first viewport 430 is not mapped on screen spaces 450and 475 and corresponds for example to another local client viewport.The second viewport 420A specifies an area in the virtual workspace 410that envelops a set of graphical targets 425 and is mapped to screenspace 450 by the local client that operates screen space 450. The thirdviewport 420B specifies an area in the virtual workspace 410 that ismapped to the screen space 475, by the local client operating the screenspace 475 in an operation to follow the viewport of the first screenspace 450. A graphical target 415 exists within the virtual workspace410 outside of the area enveloped by the first viewport 430 or thesecond viewport 420A and 42B is this example.

The first screen space 450 has a first aspect ratio, and displaysobjects having locations within the second viewport 420A (set ofgraphical targets 425). The second viewport has a matching aspect ratio.The dimension of the viewport in virtual space in the workspace can be amultiple of the aspect ratio that is different from the dimensions inpixels in the screen space 450. The multiple of the aspect ratioestablishes a zoom level in the first screen space 450, and thismultiple can be changed by the local client viewport.

The second screen space 475 has a second aspect ratio and uses aviewport 420B having the same aspect ratio as the second screen space,and encompasses the first viewport 420A. The third viewport 420B hasdimensions determined using the viewport change messages from the clientoperating the first screen space, and the parameters of the secondscreen space 475. In this example, it has the same zoom level and samecenter point as the first viewport 420A, and thus displays objectshaving locations within the second viewport 420 (set of graphicaltargets 425), and has an aspect ratio that matches the second screenspace. The graphical target 415A within the virtual workspace 410 existsoutside of 420A but overlaps viewport 420B. Thus, graphical target 415Ais not rendered on the first screen space 450 and is partially renderedon the second screen space 475. Graphic target 415B is outside theviewports 420A, 420B and 430; and thus, does not appear in the screenspace 450 or screen space 475.

FIG. 4B illustrates the second network node prior to selecting the firstnetwork node for a follow operation, and in which it has specified theviewport 430 for its local screen space 475 (in this example, the screenspace 475 is a window on a display rather than the entire display). Theset of graphical targets 435 within the first viewport 430 is renderedon the first screen space 450.

FIG. 4C illustrates the effect of logic at the second network nodeselecting the second network node to follow. This can be done by logicthat determines the viewport specifications in use at the first node forfirst screen space 450 and generates viewport specifications for thesecond node. The computation can generate local client viewportspecifications for the viewport 420B, having the same zoom level andcenter point as the viewport 420A, and that encompass the same area inthe workspace while having an aspect ratio matching the second screenspace 475 as explained above. This can be accomplished by logic thatcompares the aspect ratio (height/width) of the local screen space 475to the aspect ratio of the viewport 420A to be followed. If the aspectratio is 1 or lower, then the height is equal to or greater than thewidth. In this case, the local client viewport 420B is specified in theworkspace to have the height in the workspace of the viewport 420A to befollowed, and to have the width in the workspace determined by theaspect ratio of the local screen space 475. If the aspect ratio isgreater than 1, then the height is less than the width. In this case,the local client viewport 420B is specified to have the width in theworkspace of the viewport 420A to be followed, and to have the height inthe workspace determined by the aspect ratio of the screen space 475. Inthis way, all of the viewport 420A will fall within the viewport 420B,even if the aspect ratios do not match.

In this implementation, the participating network nodes execute agraphical user interface that includes an object such as a menu tab 482labeled “Users” in this example. The menu tab can be positioned in ornext to the screen space 475. (In this illustration the screen space 475is a small window on the display, for the purposes of the discussion ofaspect ratio matching. In other systems, the screen space can include adifferent portion of the displayable area on the display, including allof the displayable area.) The selection of the menu tab causes a usermenu to appear, where the user menu lists the identity of clients,preferably all clients, participating in the collaborative workspace. Inthis example, the participants in the collaborative workspace are aFirst Node 484 and a Second Node 486. The First Node 484 is the usernameassociated with the screen space 450. The Second Node 486 is theusername associated with the screen space 475 of FIG. 4A.

The client can include logic that upon selecting an item from the usermenu list, causes a “ye” (Volatile Event) of type “bf” (Begin Follow) tobe generated using a template form the API described below, like thefollowing:

// server <--> client [client-id, ″ve″, target-id, event-type,event-properties]  • client-id (string) the ID of the originating client • target-id (string) the ID of the target window to which this event isrelevant  • event-type (string) Volatile Event Type  • properties(object) a JSON object describing pertinent key / values for the event

Volatile Event Types that can be used in this template relating to thefollow mode can include:

bf Begin Follow: User A begins to follow User B. Used to notify User Athat user B is following. ef End Follow: User B is no longer followinguser A. Used to notify user A that user B is no longer following.

First Node 484 is identified by the client-id, and is associated withthe first screen space 450, and Second Node 486 is identified by thetarget-id and is associated with the second screen space 475 of FIG. 4A.

In this implementation, the Second Node 486 with screen space 475chooses to follow the First Node 484 with screen space 450. Second Node484's network node can create and transmit a “ye” record of type “bf” tonotify the collaboration session that it is beginning to follow theFirst Node 484. The Second Node 486 can read through its local logfinding the most recent viewport change “vc” record from First Node 484.The specifications of the First Node 486 viewport 420A from the “vc”record, are used to produce specifications for the viewport 420B of theSecond Node 484. The Second Node 484 can then use the local viewport420B to find graphical targets 425 (See, FIG. 4A) within the localviewport 420B to be rendered on the screen space 475.

Any changes in the viewport 420A, such as location or zoom level, of theFirst Node 484 are shared in the session, and are used to update thelocal viewport 420B of the Second Node 486, as long as the Second Node486 follows the First Node 484.

FIG. 5 is a flowchart that illustrates one implementation of logicexecuted at a first network node to produce a local viewport thatfollows the viewport under control of a second network node. Thetechnology disclosed allows a plurality of subject users to follow theactivities of a single target client-side network node within aworkspace.

FIG. 5 is a simplified flow diagram of a process for interpreting userinput executed by a client-side network node. The order illustrated inthe simplified flow diagram is provided for the purposes of illustrationand can be modified as suits a particular implementation. Many of thesteps, for example, can be executed in parallel. In this implementation,all users in the workspace receive all historic and ephemeral eventsthat occur within that workspace. The workspace can be, in essence,limitless, while a viewport for a client has a specific location anddimensions in the workspace. A plurality of clients can be collaboratingwithin the workspace with overlapping viewports. The clients canreceive, and log the events relating to the objects that havecoordinates outside of their viewport.

In this example, a first network node establishes communication with oneor more other network nodes 510 that are participating in acollaborative workspace. This can include logging in to a session at aserver, and communication through the server to the other network nodes,or other communication pathways.

For initialization, the first network node receives collaboration data,such as a spatial event map in the form of a log of the historic eventsthat have occurred within the workspace, or other data structure thatidentifies the location of graphical objects in the workspace. The firstnetwork node also receives, for example as a part of the log of historicevents or otherwise, a list of network nodes participating in thesession. The first network node can store the collaboration dataidentifying graphical targets having locations in a virtual workspaceused in the workspace session, the collaboration data includinglocations in a virtual workspace of respective graphical targets 520. Aspart of this initialization, the first network node can instantiate aninitial “vc” (Viewport Change) record and communicate this record toother participant clients in the workspace. The initial “vc” record canbe retrieved from the log from a previous session. The initial “vc”record can also be a default “vc” record within the workspace or can beinstantiated by some other method such logic to identify the most usedarea of the workspace. The first network node maps the local clientviewport having a location and dimensions within the virtual workspaceto a local client screen space 530. In order to set the “follow” flag, afirst subject user associated with the first network node is provided auser interface displaying a list of participants in the session at othernetwork nodes 540. The first subject user can then select a target userfrom the list of identified users. The first network node creates a“ve:bf” record and communicates the record to the participating networknodes of the workspace, which can include the network node of theselected user. The first subject network node is now “following” theselected network node.

The first network node continues to receive messages from otherparticipants in the session, including messages containing a location inthe workspace of a participant client viewport 550 in use at theselected other participant client. The first network node only rendersthose graphical targets that appear within the coordinates of theviewport of the selected user. As the selected other participant clientchanges the viewport of the selected network node, the first networknode can update the location of the local client viewport to theidentified location of the participant client viewport in use at theselected other participant client 560.

The first network node can then render on the screen space graphicaltargets having locations within the updated local client viewport 570.

The technology disclosed can render the contents of the viewport ontothe local screen space accounting for possible differences in attributessuch as aspect ratio, height, and width of the local screen space.Changes to the virtual workspace, such as the addition, deletion, ormodification of graphical targets that intersect the viewport of theselected user are also rendered on the screen space of the first networknode.

A third user can also set a follow flag within their network node tofollow the first network node, the selected user's network node, or someother network node. At any point, the first user can unset the followflag and return to their original viewport.

In this example, a plurality of clients within a workspace can be asubject network node or a target network node for the “follow method”.In one implementation, a network node can follow multiple targets, andcan both follow a target in one viewport as well as collaborate within asecond viewport simultaneously.

FIG. 6 illustrates a message map between a first client-side networknode, a server-side network node, and a second client-side network nodefor the first client-side network node to follow the second client-sidenetwork node.

Initially, in this implementation, a second client-side network node 619is participating in a collaborative workspace. The second client-sidenetwork node 619 can create events that are shared with other networknodes through a spatial event map on a server-side network node 615. Thesecond client-side network node 619 can also receive events from otherclient-side network nodes through the server-side network node 615. Thetransmission of events occurs through a communication of events 620between one or more client-side network nodes and the server-sidenetwork node. The server-side node distributes the events to otherparticipating network nodes in this example.

In this example, the first client-side network node 611 joins thecollaborative workspace by establishing communications with theserver-side network node 630. The server-side network node sends thecollaboration data, including a user list, the viewport change recordsof the network nodes, the spatial event map, and location markers, tothe first client-side network node 640. The first client-side networknode then stores the collaboration data to a local log. The firstclient-side network node 611 sets an initial viewport, as described inFIG. 5 , then discovers the event records of graphical targets withinthe local log that intersect its viewport. The graphical targets thatintersect the viewport are then rendered on the screen space.

The first client-side network node 611 and the second client-sidenetwork node 619 can both create, transmit, and receive events withinthe workspace, and can view events that have occurred within theirviewports. Events can be communicated to all participating network nodesthrough the server-side network node 650. The technology disclosedallows a first user to follow a second user. In this example, a firstclient-side network node 611 can select from the list of users in itslocal log the identity of a target user working on a second client-sidenetwork node 619. This selection causes the first client-side networknode to create a begin following “ve:bf” record, which can becommunicated to the second client-side network node through theserver-side network node 660. Events are still communicated to allparticipating network nodes. However, the first client-side network nodeproduces a local client viewport that encompasses the viewport specifiedat the second client-side network node, and only displays those eventsthat intersect the updated local client viewport, and in effect followsthe viewport of the second client-side network node. Events can begenerated by the second client-side network node 619, or by a thirdclient-side network node not shown in FIG. 6 . However, in this example,the first client-side network node 611 does not generate events thatchange the displayable contents of the workspace while in “follow me”mode, but continues to receive events created by other network nodes670.

In this example, the second client-side network node 619 can change itsviewport. The change in viewport causes the second client-side networknode to generate a new viewport change “vc” record and transmit thatrecord to other participants in the workspace 680. The first client-sidenetwork node, while following the second client-side network node,changes its local client viewport to match, then renders the graphicaltargets from its log that intersects the new local client viewport.

In another viewport finding technology, events that set location markershaving locations in the workspace can be shared in a spatial event map,or other kind of collaboration data structure. A first user at a firstnetwork node can select a location marker set by any participant at anytime. A location marker can be used as a record of where a second userhas been, rather than where they are now.

FIGS. 7A, 7B, and 7C illustrate a screen space with examples of alocation marker (located within the current viewport of a client), andsome of the events that can occur with them. FIG. 7A illustrates aplurality of location markers in a screen space including a firstlocation marker 721, a second location marker 723, a third locationmarker 725, and a graphical target 751.

FIG. 7B illustrates the objects in FIG. 7A. FIG. 7B also shows the nameof the third location marker 725 after a user has clicked one time onthe location marker with a mouse, which causes the name of the thirdlocation marker to appear. In this example, the name of the locationmarker 725 is “Location A”.

FIG. 7C shows a zoom into the workspace caused by changing thedimensions of the current viewport (with fixed the dimensions of thescreen space), with the second location marker 723 becoming the centercoordinate (0,0) of the screen space after a pointer or touch based userinput gesture, such as for example double click on the second locationmarker. The double click gesture causes the local client to compute anew viewport specification for the local viewport, centered on thelocation in the workspace of the double click, which is on the locationmarker in this example, and having a zoom level increased as a result ofthe double click. In this example, the name of the third location marker725 has disappeared, and the viewport dimensions are changed to define asmaller space within the workspace, where the center of the viewport isthe location of the double click, i.e. second location marker 723, andthe center of the screen space is also the location of the double click.The first location marker 721 is no longer within the viewport, and thegraphical target of third location marker 725 is only partially visiblewithin the screen space, where part of the third location marker isoutside of the viewport. The viewport no longer contains all of thegraphical target 751, but only the portion of the graphical target thatfits within the viewport.

FIGS. 8A and 8B illustrate a graphical user interface having constructsused for selecting a location marker from a menu. FIG. 8A illustrates aplurality of objects including a first location marker 821 for LocationA, a second location marker 823 for Location B, a third location marker825 for Location C, and a graphical target 851. Also illustrated is alocation icon 803. Clicking on the location icon 803 in the menu barcauses a location menu 801 to appear, which can have zero or moreentries. In this example, the location menu 801 has 5 entries: add alocation, change viewport to a preset origin, and three locationmarkers. There can be other location markers in the menu that are notrepresented in the screen space. Selecting an entry on the menu in thisexample, as illustrated in FIG. 8B, causes an update to the viewportspecifications for the local viewport that in effect results in movingthe selected location marker to the center coordinate (e.g. 0,0) of thescreen space, while maintaining the zoom level in effect at the time ofthe selection. This user interface displaying a list of location markersalso displays a selectable entry (“origin”) for default location orpreset origin location for the workspace, in addition to a list oflocation markers. The client includes logic to update the location ofthe local client viewport to the default location upon selection of the“origin” selectable entry, while maintaining the zoom level in effect atthe time of the selection.

As illustrated in FIG. 8B, within the location menu 801 is a selectableitem labeled Location A 814. Selecting Location A 814 causes theviewport to use the first location marker 821 as the center of theviewport, and for the contents of the screen space to appear to move upand to the right. The second location marker 823 and graphical target851 are now only partially visible in the screen space, and the thirdlocation marker 825 is no longer visible within the screen space. Inthis example, the local client viewport moves in the workspace, withoutchanging size. As a result, zoom level remains constant.

The location marker names are searchable tags that can reflect contentof the material having locations in the virtual workspace. In aworkspace having large numbers of location markers, the user interfacecan include search function base on querying the location marker tags.

FIGS. 9A, 9B, and 9C illustrate an example of adding a location markerto a virtual workspace. FIG. 9A illustrates a plurality of objectsincluding a first location marker 921, a second location marker 923, athird location marker 925, and a graphical target 951. Also illustratedis a location icon 903 and a location menu 901, which can have zero ormore entries. In this example, the first entry in the location menu 901is Add Location 911. Selecting the Add Location 911 entry can invoke thesub-menu items illustrated in FIG. 9B, which include a location markertype 915 (used to select the graphical target of the create locationmarker event to be displayed), a field for location marker name 916, anda Create Marker button 917. The creation of a new location marker caninclude a selection of a marker type 915. In this implementation, themarker types available are color coded to indicate different categoriesof location markers. A next step is to enter a value in the field forlocation marker name 916, which, in this example, is “New LocationMarker”. The final step is to select the Create Marker button 917. Asillustrated in FIG. 9C, this selection causes the sub-menu to disappear,and for a new location marker 927 named “New Location Marker” to appearin the middle of the screen space. An “he:markercreate” record is thencreated and distributed to other participating network nodes:

  [client-id, ″he″, new-widget-id, ″markercreate″,{ ″creationTime″:1387565966,  ″name″:″New Location Marker″,  ″y″:1828, ″x″:−875,  ″color″:0 }]

The client running in the network node that creates a new locationmarker, includes logic to send messages to other network nodes, themessages identifying events including creation or movement of a locationmarker. Also, the logic in the client running at the network node, andat other network nodes in the collaboration system, s responsive toreceipt of a message carrying the marker create or marker move eventsfrom another (second) network node to update the list of locationmarkers.

FIG. 10 is a flowchart that illustrates the selection of a locationmarker from a menu. FIG. 10 is a simplified flow diagram of a processfor interpreting user input executed by a client-side network node. Theorder illustrated in the simplified flow diagram is provided for thepurposes of illustration and can be modified as suits a particularimplementation. Many of the steps for example, can be executed inparallel. In this implementation, all users in the workspace receive allhistoric and ephemeral events that occur within that workspace. Theworkspace can be, in essence, limitless, while a viewport for a clientcan include the contents of a portion of the workspace that intersectsthe client's viewport. A plurality of clients can be collaboratingwithin the workspace with overlapping viewports. The clients can receiveand log the objects defined by event records that have coordinatesoutside of their viewport.

In this example, a first network node establishes communication with oneor more other network nodes as a participant client in a collaborativeworkspace session 1010. The first network node stores at least part of alog of events relating to graphical targets, including creation ormovement of location markers. The targets of the events have locationsin a virtual workspace used in the workspace session, entries in the logincluding a location in the virtual workspace of a graphical target ofan event, an action related to the graphical target, a time of theevent, and a target identifier of the graphical target 1020. The firstnetwork node maps a local client viewport to a local client screen spacein the physical display space at the first network node 1030. Thenetwork node provides a user interface displaying a list of locationmarkers in the workspace in the session, a location marker having amarked location in the workspace, and for receiving input indicating aselected location marker from the list 1040.

In response to selection of a selected location marker, the localnetwork node updates the location of the local client viewport to themarked location 1050 and renders on the screen space graphical targetshaving locations within the updated local client viewport 1060.

FIG. 11 illustrates a message map for communication of location markersbetween network nodes. FIG. 11 illustrates a message map between a firstclient-side network node, a server-side network node, and a secondclient-side network node for the first client-side network node tofollow the second client-side network node.

Initially, in this implementation, a second client-side network node1119 is participating in a collaborative workspace. The secondclient-side network node 1119 can create events that are shared withother network nodes through a spatial event map on a server-side networknode 1115. The second client-side network node 1119 can also receiveevents from other client-side network nodes through the server-sidenetwork node 1115. The transmission of events occurs through acommunication of events 1120 between one or more client-side networknodes and the server-side network node.

In this example, the first client-side network node 1111 joins thecollaborative workspace by establishing communication with theserver-side network node 1130. The server-side network node sends thecollaboration data, including graphical targets such as locationmarkers, to the first client-side network node 1140. The firstclient-side network node then stores the collaboration data to a locallog. The first client-side network node 1111 sets an initial viewport,as described in FIG. 5 , then discovers the event records of graphicaltargets within the local log that intersect its viewport. The graphicaltargets of the events that intersect the viewport are then rendered onthe screen space.

The first client-side network node 1111 and the second client-sidenetwork node 1119 can both create, transmit, and receive events withinthe workspace, and can view events that have occurred within theirviewports. Events can be communicated to all participating network nodesthrough the server-side network node 1150. The technology disclosedallows a first user to create a location marker that is shared with asecond user. In this example, a first client-side network node 1111 canadd a location marker, as illustrated in FIG. 9 . This causes an“he:markercreate” record to be created, which can be communicated to thesecond client-side network node through the server-side network node1160. From that point on, the location marker is communicated to otherparticipants in the workspace 1170 where it can be selected. Uponinvoking a location marker, and updating is local viewport as a result,a participant node can send a viewport change “vc” event to the otherparticipants.

FIG. 12 illustrates example aspects of a digital display collaborationenvironment. In the example, a plurality of users 1201 a-h (collectively1201) may desire to collaborate with each other in the creation ofcomplex images, music, video, documents, and/or other media, allgenerally designated in FIG. 12 as 1203 a-d (collectively 1203). Theusers in the illustrated example use a variety of devices configured aselectronic network nodes, in order to collaborate with each other, forexample a tablet 1202 a, a personal computer (PC) 1202 b, and many largeformat displays 1202 c, 1202 d, 1202 e (collectively devices 1202). Inthe illustrated example the large format display 1202 c, which issometimes referred to herein as a “wall”, accommodates more than one ofthe users, (e.g. users 1201 c and 1201 d, users 1201 e and 1201 f, andusers 1201 g and 1201 h). The user devices, which are referred to asclient-side network nodes, have displays on which a screen space isallocated for displaying events in a workspace. The screen space for agiven user may comprise the entire screen of the display, a subset ofthe screen, a window to be displayed on the screen and so on, such thateach has a limited area or extent compared to the virtually unlimitedextent of the workspace.

FIG. 13 illustrates additional example aspects of a digital displaycollaboration environment. As shown in FIG. 12 , the large formatdisplays 1202 c, 1202 d, 1202 e sometimes referred to herein as “walls”are controlled by respective client-side, communication networks 1304,which in turn are in network communication with a central collaborationserver 1305 configured as a server-side physical network node or nodes,which has accessible thereto a database 1306 storing spatial event mapstacks for a plurality of workspaces.

As used herein, a physical network node is an active electronic devicethat is attached to a network, and is capable of sending, receiving, orforwarding information over a communication channel. Examples ofelectronic devices which can be deployed as network nodes, include allvarieties of computers, workstations, laptop computers, handheldcomputers and smart phones. As used herein, the term “database” does notnecessarily imply any unity of structure. For example, two or moreseparate databases, when considered together, still constitute a“database” as that term is used herein.

The application running at the collaboration server 1305 can be hostedusing Web server software such as Apache or nginx, or a runtimeenvironment such as node.js. It can be hosted for example on virtualmachines running operating systems such as LINUX. The server 1305 isillustrated, heuristically, in FIG. 13 as a single computer. However,the server architecture can involve systems of many computers, eachrunning server applications, as is typical for large-scale cloud-basedservices. The server architecture includes a communication module, whichcan be configured for various types of communication channels, includingmore than one channel for each client in a collaboration session. Forexample, with near-real-time updates across the network, client softwarecan communicate with the server communication module using amessage-based channel, based for example on the Web Socket protocol. Forfile uploads as well as receiving initial large volume workspace data,the client software can communicate with the server communication modulevia HTTPS. The server can run a front-end program written for example inJavaScript served by Ruby-on-Rails, support authentication/authorizationbased for example on Oauth, and support coordination among multipledistributed clients. The server communication module can include amessage-based communication protocol stack, such as a Web Socketapplication, that performs the functions of recording user actions inworkspace data, and relaying user actions to other clients asapplicable. This system can run on the node.JS platform for example, oron other server technologies designed to handle high-load socketapplications.

The database 1306 stores, for example, a digital representation ofworkspace data sets for a spatial event map of each session where theworkspace data set can include or identify events related to objectsdisplayable on a display canvas. A workspace data set can be implementedin the form of a spatial event stack, managed so that at leastpersistent spatial events (called historic events) are added to thestack (push) and removed from the stack (pop) in a first-in-last-outpattern during an undo operation. There can be workspace data sets formany different workspaces. A data set for a given workspace can beconfigured in a database, or as a machine-readable document linked tothe workspace. The workspace can have unlimited or virtually unlimiteddimensions. The workspace data includes event data structuresidentifying objects displayable by a display client in the display areaon a display wall and associates a time and a location in the workspacewith the objects identified by the event data structures. Each device1202 displays only a portion of the overall workspace. A display wallhas a display area for displaying objects, the display area being mappedto a corresponding area in the workspace that corresponds to a viewportin the workspace centered on, or otherwise located with, a user locationin the workspace. The mapping of the display area to a correspondingviewport in the workspace is usable by the display client to identifyobjects in the workspace data within the display area to be rendered onthe display, and to identify objects to which to link user touch inputsat positions in the display area on the display.

The server 1305 and database 1306 can constitute a server-side networknode, including memory storing a log of events relating to graphicaltargets having locations in a workspace, entries in the log including alocation in the workspace of the graphical target of the event, a timeof the event, and a target identifier of the graphical target of theevent. The server can include logic to establish links to a plurality ofactive client-side network nodes, to receive messages identifying eventsrelating to modification and creation of graphical targets havinglocations in the workspace, to add events to the log in response to saidmessages, and to distribute messages relating to events identified inmessages received from a particular client-side network node to otheractive client-side network nodes.

The logic in the server 1305 can comprise an application programinterface, including a specified set of procedures and parameters, bywhich to send messages carrying portions of the log to client-sidenetwork nodes, and to receive messages from client-side network nodescarrying data identifying events relating to graphical targets havinglocations in the workspace.

Also, the logic in the server 1305 can include an application interfaceincluding a process to distribute events received from one client-sidenetwork node to other client-side network nodes.

The events compliant with the API can include a first class of event(history event) to be stored in the log and distributed to otherclient-side network nodes, and a second class of event (ephemeral event)to be distributed to other client-side network nodes but not stored inthe log.

The server 1305 can store workspace data sets for a plurality ofworkspaces and provide the workspace data to the display clientsparticipating in the session. The workspace data is then used by thecomputer systems 1310 with appropriate software 1312 including displayclient software, to determine images to display on the display, and toassign objects for interaction to locations on the display surface. Theserver 1305 can store and maintain a multitude of workspaces, fordifferent collaboration sessions. Each workspace can be associated witha group of users and configured for access only by authorized users inthe group.

In some alternatives, the server 1305 can keep track of a “viewport” foreach device 1202, indicating the portion of the canvas viewable on thatdevice, and can provide to each device 1202 data needed to render theviewport.

Application software running on the client device responsible forrendering drawing objects, handling user inputs, and communicating withthe server can be based on HTML5 or other markup based procedures andrun in a browser environment. This allows for easy support of manydifferent client operating system environments.

The user interface data stored in database 1306 includes various typesof objects including graphical constructs, such as image bitmaps, videoobjects, multi-page documents, scalable vector graphics, and the like.The devices 1202 are each in communication with the collaboration server1305 via a communication network 1304. The communication network 1304can include all forms of networking components, such as LANs, WANs,routers, switches, WiFi components, cellular components, wired andoptical components, and the internet. In one scenario two or more of theusers 1201 are located in the same room, and their devices 1202communicate via WiFi with the collaboration server 1305. In anotherscenario two or more of the users 1201 are separated from each other bythousands of miles and their devices 1202 communicate with thecollaboration server 1305 via the internet. The walls 1202 c, 1202 d,1202 e can be multi-touch devices which not only display images, butalso can sense user gestures provided by touching the display surfaceswith either a stylus or a part of the body such as one or more fingers.In some embodiments, a wall (e.g. 1202 c) can distinguish between atouch by one or more fingers (or an entire hand, for example), and atouch by the stylus. In an embodiment, the wall senses touch by emittinginfrared light and detecting light received; light reflected from auser's finger has a characteristic which the wall distinguishes fromambient received light. The stylus emits its own infrared light in amanner that the wall can distinguish from both ambient light and lightreflected from a user's finger. The wall 1202 c may, for example, be anarray of Model No. MT553UTBL MultiTaction Cells, manufactured byMultiTouch Ltd, Helsinki, Finland, tiled both vertically andhorizontally. In order to provide a variety of expressive means, thewall 1202 c is operated in such a way that it maintains “state.” Thatis, it may react to a given input differently depending on (among otherthings) the sequence of inputs. For example, using a toolbar, a user canselect any of a number of available brush styles and colors. Onceselected, the wall is in a state in which subsequent strokes by thestylus will draw a line using the selected brush style and color.

In an illustrative embodiment, a display array can have a displayablearea usable as a screen space totaling on the order of 6 feet in heightand 30 feet in width, which is wide enough for multiple users to standat different parts of the wall and manipulate it simultaneously.

FIGS. 14A-14F represent data structures which can be part of workspacedata maintained by a database at the collaboration server 1305. In FIG.14A, an event data structure is illustrated. An event is an interactionwith the workspace data that can result in a change in workspace data.Thus, an event can include an event identifier, a timestamp, a sessionidentifier, an event type parameter, the client identifier as client-id,and an array of locations in the workspace, which can include one ormore for the corresponding event. It is desirable, for example, that thetimestamp have resolution on the order of milliseconds or even finerresolution, in order to minimize the possibility of race conditions forcompeting events affecting a single object. Also, the event datastructure can include a UI target, which identifies an object in theworkspace data to which a stroke on a touchscreen at a client display islinked. Events can include style events, which indicate the displayparameters of a stroke, for example. The events can include a text typeevent, which indicates entry, modification or movement in the workspaceof a text object. The events can include a card type event, whichindicates the creation, modification or movement in the workspace of acard type object. The events can include a stroke type event whichidentifies a location array for the stroke, and display parameters forthe stroke, such as colors and line widths for example. Events caninclude begin follow events and viewport change events. Events caninclude marker create events.

Events can be classified as persistent history events and as ephemeralevents. Processing of the events for addition to workspace data andsharing among users can be dependent on the classification of the event.This classification can be inherent in the event type parameter, or anadditional flag or field can be used in the event data structure toindicate the classification.

A spatial event map can include a log of events having entries forhistory events, where each entry comprises a structure such asillustrated in FIG. 14A. A server-side network node includes logic toreceive messages carrying ephemeral and history events from client-sidenetwork nodes, and to send the ephemeral events to other client-sidenetwork nodes without forwarding them to a server at which events areadded as corresponding entries in the log, and to send history events tothe other client-side network nodes while forwarding them to a server atwhich events are added as corresponding entries to the log.

FIG. 14B illustrates a card data structure. The card data structure canprovide a cache of attributes that identify current state informationfor an object in the workspace data, including a session identifier, acard type identifier, an array identifier, the client identifier,dimensions of the cards, type of file associated with the card, and asession location within the workspace.

FIG. 14C illustrates a data structure which consolidates a number ofevents and objects into a cacheable set called a chunk. The datastructure includes a session ID, and identifier of the events includedin the chunk, and a timestamp at which the chunk was created.

FIG. 14D illustrates the data structure for links to a userparticipating in a session in a chosen workspace. This data structurecan include an access token, the client identifier for the sessiondisplay client, the user identifier linked to the display client, aparameter indicating the last time that a user accessed a session, andexpiration time and a cookie for carrying various information about thesession. This information can, for example, maintain a current locationwithin the workspace for a user, which can be used each time that a userlogs in to determine the workspace data to display at a display clientto which the login is associated.

FIG. 14E illustrates a display array data structure which can be used inassociation with large-format displays that are implemented by federateddisplays, each having a display client. The display clients in suchfederated displays cooperate to act as a single display. The workspacedata can maintain the display array data structure which identifies thearray of displays by an array ID and identifies the session position ofeach display. Each session position can include an x-offset and ay-offset within the area of the federated displays, a sessionidentifier, and a depth.

The system can encrypt communications with client-side network nodes andcan encrypt the database in which the spatial event maps are stored.Also, on the client-side network nodes, cached copies of the spatialevent map are encrypted in some embodiments, to prevent unauthorizedaccess to the data by intruders who gain access to the client-sidecomputers.

FIG. 14F illustrates a Global Session Activity Table (GSAT) used to mapactive clients to active workspaces. The data structure includes aworkspace name, a device type, a client ID, a session ID, and actortype, and an actor ID.

FIG. 15 is a simplified block diagram of a computer system, or networknode, which can be used to implement the client-side functions (e.g.computer system 1310) or the server-side functions (e.g. server 1305) ina distributed collaboration system. A computer system typically includesa processor subsystem 1514 which communicates with a number ofperipheral devices via bus subsystem 1512. These peripheral devices mayinclude a storage subsystem 1524, comprising a memory subsystem 1526 anda file storage subsystem 1528, user interface input devices 1522, userinterface output devices 1520, and a network interface subsystem withina communication module 1516. The input and output devices allow userinteraction with the computer system. Communication module 1516 providesphysical and communication protocol support for interfaces to outsidenetworks, including an interface to communication network 1304, and iscoupled via communication network 1304 to corresponding communicationmodules in other computer systems. Communication network 1304 maycomprise many interconnected computer systems and communication links.These communication links may be wireline links, optical links, wirelesslinks, or any other mechanisms for communication of information, buttypically it is an IP-based communication network, at least at itsextremities. While in one embodiment, communication network 1304 is theInternet, in other embodiments, communication network 1304 may be anysuitable computer network.

The physical hardware components of network interfaces are sometimesreferred to as network interface cards (NICs), although they need not bein the form of cards: for instance, they could be in the form ofintegrated circuits (ICs) and connectors fitted directly onto amotherboard, or in the form of macrocells fabricated on a singleintegrated circuit chip with other components of the computer system.

User interface input devices 1522 may include a keyboard, pointingdevices such as a mouse, trackball, touchpad, or graphics tablet, ascanner, a touch screen incorporated into the display (including thetouch sensitive portions of large format digital display 1202 c), audioinput devices such as voice recognition systems, microphones, and othertypes of tangible input devices. In general, use of the term “inputdevice” is intended to include all possible types of devices and ways toinput information into the computer system or onto communication network1304.

User interface output devices 1520 may include a display subsystem, aprinter, a fax machine, or non-visual displays such as audio outputdevices. The display subsystem can include a cathode ray tube (CRT), aflat panel device such as a liquid crystal display (LCD), a projectiondevice, or some other mechanism for creating a visible image. In theembodiment of FIG. 12 , it includes the display functions of largeformat digital display 1202 c. In general, use of the term “outputdevice” is intended to include all possible types of devices and ways tooutput information from the computer system to the user or to anothermachine or computer system.

Storage subsystem 1524 stores the basic programming and data constructsthat provide the functionality of certain embodiments of the technologydisclosed. The storage subsystem 1524 includes computer programinstructions implementing a spatial event map collaboration systemclient, or a spatial event map collaboration system server, and caninclude logic for modules such as “follow me” and “location marker”.

The storage subsystem 1524 when used for implementation of server sidenetwork-nodes, comprises a product including a non-transitory computerreadable medium storing a machine readable data structure including aspatial event map which locates events in a workspace, wherein thespatial event map includes a log of events, entries in the log having alocation of a graphical target of the event in the workspace and a time.Also, the storage subsystem 1524 comprises a product includingexecutable instructions for performing the procedures described hereinassociated with the server-side network node.

The storage subsystem 1524 when used for implementation of client sidenetwork-nodes, comprises a product including a non-transitory computerreadable medium storing a machine readable data structure including aspatial event map in the form of a cached copy as explained below, whichlocates events in a workspace, wherein the spatial event map includes alog of events, entries in the log having a location of a graphicaltarget of the event in the workspace and a time. Also, the storagesubsystem 1524 comprises a product including executable instructions forperforming the procedures described herein associated with theclient-side network node.

For example, the various modules implementing the functionality ofcertain embodiments of the technology disclosed may be stored in storagesubsystem 1524. These software modules are generally executed byprocessor subsystem 1514.

Memory subsystem 1526 typically includes a number of memories includinga main random-access memory (RAM) 1530 for storage of instructions anddata during program execution and a read only memory (ROM) 1532 in whichfixed instructions are stored. File storage subsystem 1528 providespersistent storage for program and data files, and may include a harddisk drive, a floppy disk drive along with associated removable media, aCD ROM drive, an optical drive, or removable media cartridges. Thedatabases and modules implementing the functionality of certainembodiments of the technology disclosed may have been provided on acomputer readable medium such as one or more CD-ROMs and may be storedby file storage subsystem 1528. The host memory subsystem 1526 contains,among other things, computer instructions which, when executed by theprocessor subsystem 1514, cause the computer system to operate orperform functions as described herein. As used herein, processes andsoftware that are said to run in or on “the host” or “the computer,”execute on the processor subsystem 1514 in response to computerinstructions and data in the host memory subsystem 1526 including anyother local or remote storage for such instructions and data.

Bus subsystem 1512 provides a mechanism for letting the variouscomponents and subsystems of a computer system communicate with eachother as intended. Although bus subsystem 1512 is shown schematically asa single bus, alternative embodiments of the bus subsystem may usemultiple busses.

The computer system itself can be of varying types including a personalcomputer, a portable computer, a workstation, a computer terminal, anetwork computer, a television, a mainframe, a server farm, or any otherdata processing system or user device. In one embodiment, a computersystem includes several computer systems, each controlling one of thetiles that make up the large format display 1202 c. Due to theever-changing nature of computers and networks, the description ofcomputer system 1310 depicted in FIG. 15 is intended only as a specificexample for purposes of illustrating the preferred embodiments of thetechnology disclosed. Many other configurations of the computer systemare possible having more or less components than the computer systemdepicted in FIG. 15 . The same components and variations can also makeup each of the other devices 1202 in the collaboration environment ofFIG. 13 , as well as the collaboration server 1305 and display database1306.

API

Aspects of an application program interface API supporting use ofspatial event maps, with follow me and location marker functions are setout here for the purposes of example of technology to implement thetechniques described herein.

Socket Requests Server (Web Sockets)—used for updating clients withrelevant data (new strokes, cards, clients, etc.) once connected. Alsohandles the initial connection handshake.

Service Requests Server (HTTPS/REST)—used for cacheable responses, aswell as posting data (i.e. images and cards)

Client-side network nodes are configured according to the API andinclude corresponding socket requests clients and service requestsclients.

All messages are individual UTF-8 encoded JSON arrays. For example:

[sender-id, message-type, ...]  • sender-id the ID of the client sendingthe message, or ″−1″ if the   message originates with the server.Sender-ids are unique among all clients   connected to the server.  •message-type a short code identifying the remaining arguments and the  intended action.

Establishing a Connection

Clients use the Configuration Service to retrieve the configurationinformation for the environment. The socket server URL is provided bythe ws_collaboration_service_address key.

1) To Open the WebSocket and Join a Specific Workspace

<collaboration_service_address>/<workspaceId>/socket?device=<device>array=<array_na me>  • workspaceId (string)  the id of the workspace to join  • device(string) a device type. Supported values: wall, other  • array (string)an identifier for multiple screen instances. (optional)

2) To Join the Lobby

The lobby allows a client to open a web socket before it knows thespecific workspace it will display. The lobby also provides a 5-digitPIN which users can use to send a workspace to the wall from theirpersonal device (desktop/ios).

<collaboration_service_address>/lobby/socket?device=<device>&array=<array_name> • device (string) a device type. Supported values: wall, other  • array(string) an identifier for multiple screen instances. (optional)

3) Server Response

When a client establishes a new web-socket connection with the server,the server first chooses a unique client ID and sends it in an “id”message to the client with the unique client ID.

4) Message Structure

The first element of each message array is a sender-id, specifying theclient that originated the message. Sender-ids are unique among allsessions on the server. The id and cr messages sent from the server tothe client have their sender-id set to a default value, such as −1. Thesecond element of each message array is a two-character code. This codedefines the remaining arguments in the array as well as the intendedaction. Messages sent with a sender-id of −1 are messages that originatefrom the server.

Message Types

The following messages types are officially supported. Since Spring 2013there has been an effort to use he and ye when possible instead ofadding new top level message types.

 1) cs Change Session  2) echo Echo  3) error Error  4) id Client Id  5)jr Join Room  6) rl Room List  7) un Undo  8) up User Permissions  9) vcViewport Change 10) he History Event 11) ve Volatile Event 12)disconnect Disconnect 13) ls List Streams 14) bs Begin Stream 15) es EndStream 16) ss Stream State 17) oid Object Id Reservation

1) cs Change Session

Inform a client or siblings in a display array that the workspace haschanged. The server sends this message to the client when it receivesrequest to send a workspace to a wall.

  // server --> client [sender-id, ″cs″, workspaceId]  sender-id always−1 (indicating the server initiated the message)  workspaceId (string)is the id of the workspace to switch to

2) echo Echo

Echos an optional body back to the originating client. Used to verifythat the socket connection and the server are still healthy.

  // client --> server [sender-id, ″echo″, ″foo″, ″bar″...] // server--> client [−1, ″echo″, ″foo″, ″bar″...]

After “echo” the message can take any arguments. They will all be echoedback to the client unaltered if the service and the client's socketconnection are healthy. When using the echo message to verify sockethealth we recommend the following:

-   -   Wait at least 5 seconds between echo messages    -   Require 2 consecutive echo failures before assuming network or        server problems

This message was added to the protocol because the currentimplementation of WebSockets in Chrome and other supported browsers donot correctly change readyState or fire onclose when network connectiondies.

3) error Error

Informs clients of an error on the server side.

// server -> client [″−1″, ″error″, target-id, message]  target-id theguid for the object in the session that the error affects  message amessage about the error.

This message is only sent by the server and currently only used when anupload fails during asynchronous processing.

4) id Client Id

The server sends this message when the client connects to the socket.Clients are required to store the assigned client ID for use insubsequent socket requests.

  // server --> client [″−1″, ″id″, client-id]  client-id (string) theID of the newly-joined client

5) jr Join Room

Rooms are communication channels that a client can subscribe to. Therecan be channels for specific workspaces (sessions), for display arrays,or potentially for other groups of clients. The server repeats/sendsmessages to all the clients connected to a specific room as eventshappen in that room. A client joins a room to get messages for thatdisplay array or workspace (session). There are several types of joinroom requests.

General jr

Join any room if you know the id.

// server <-- client [sender-id, ″jr″, room-id, [data]]  room-id cancontain one of lobby or workspace id  data is a wildcard set ofarguments, which should be used to initialize the room.

Lobby jr

Joins the lobby channel. Used by clients that wish to keep a web socketopen while not displaying a workspace.

  // server <-- client [sender-id, ″jr″, ″lobby″]

Session jr

Joins the room for a specific workspace (session).

  // server <-- client [sender-id, ″jr″, ″session″, workspace-id] workspace-id the id of the workspace (workspace)

Array jr

Joins the room for a specific display array.

  // server <-- client [sender-id, ″jr″, ″array″, {  arrayId:″myArrayId″,  x: 0,  y: 0,  width: 1920,  height: 1080 }]   arrayID(string) id of the display array   x (integer) x offset of this display  y (integer) y offset of this display   width (integer)  width of thisdisplay   height (integer) height of this display

Room Join Response:

The server responds to successful room join (jr) messages with a roommessage.

General Room

  // server --> client [″−1″, ″room″, [room-id], [databag]]  • room-idcontains one of: lobby or workspace  • databag is a room-specific bag ofvariables:

Lobby Room

  // server --> client [″−1″, ″room″, ″lobby″, {pin: pin-code}]  • pincontaining the pin for wall authentication

Session Room

// server --> client[″−1″,″room″,″session″,{″uid″:″SU5DVpxbfnyGCesijBou″,″name″:″Dec 16  Release″,″sharing_link″:″https://portal.Bluescape.com/sessions/1357/shares″}]′′ *′uid′ the id of the workspace * ′name′ the name of the workspace to showin the client * ′sharing_link′ a link to the portal page where a usercan share this  workspace with others

6) rl Room List

Informs the client of the room memberships. Room memberships includeinformation regarding clients visiting the same room as you.

// server --> client [″−1″, ″rl″, roomMembershipList]  •roomMembershipList (array of room membership objects) A room membershipobject is a hash with the following keys  • name User or device name  •device_type The type of device the user is on, currently wall or other.  (Deprecated)  • clientId The clientId of this device  • clientType Thetype of client (browser, ipad, or wall)  • viewport (optional) If theclient provides a viewport rect the server will repeat it   to allclients.

7) un Undo

Undoes the last undo-able action (move, set text, stroke, etc).

  // server <-- client [sender-id, ″un″, region-id] // server --> client[client-id, ′undo′, target-id, removedEventId]Undo Example: Move a Window and then Undo that Move

The following example shows a move, and how that move is undone.

// Client sends move[″5122895cff31fe3509000001″,″he″,″5122898bff31fe3509000002″,″position″,{ ″rect″:[257,357,537,517],  ″order″:2 }] // Server response[″5122895cff31fe3509000001″,″he″,″5122898bff31fe3509000002″,″5122898efde0f33509  000008″,″position″,{  ″rect″:[257,357,537,517]  ,″order″:2 }] //Client sends undo [<clientId>, ′un′, <canvasRegionId>][″5122895cff31fe3509000001″,″un″,null] // Server response //[<clientId>, ′undo′, <targetId>, <removedMessageId>][″−1″,″undo″,″5122898bff31fe3509000002″,″5122898efde0f33509000008″]

The server removes the history event from the workspace history andnotifies all clients subscribed to the room that this record will nolonger be a part of the workspace's historical timeline. Future requestsof the history via the HTTP API will not include the undone event (untilwe implement redo).

8) up User Permissions

Gets the permissions that the current user has for this workspace. Onlyrelevant when a client is acting as an agent for a specific user notrelevant when using public key authentication (walls).

// server --> client [sender-id, ″up″, permissions]  • Permissions ahash of permission types and true/false to indicate if the  authenticated user has that permission. Currently the only permissionis   ″can_share″ indicating users who can share the workspace withothers.

9) vc Viewport Change

Updates other clients in a session that one client's viewport haschanged. This is designed to support the “jump to user's view” and“follow me” features. Clients send a viewport change “vc” event uponentering a session for the first time. This ensures that other clientswill be able to follow their movements. When processing incomingviewport change “vc” events, clients keep a cache of viewports, keyed byclient ID. This is in order to handle occasions where room listmembership (rl) events with missing viewports arrive after associated VCevents. A change in a target viewport to a revised target viewport caninclude a change in the size of the viewport in one or the otherdimension or both, which does not maintain the aspect ratio of theviewport. A change in a target viewport can also include a change in thepage zoom of the viewport. When subject client-side viewports in “jumpto user's view” or “follow-me” mode receive a first ‘vc’ record, it isan instruction for mapping a displayable area of the subject client-sideviewport to the area of a target viewport. A subsequent ‘vc’ recordresults in a remapped displayable area of the subject client-sideviewport to the target viewport. When the “jump to user's view” or“follow me” feature is disabled, the subject client-side viewportreturns to its prior window.

// server <--> client [sender-id, ″vc″, viewport-rect]  • viewport-rectan array in the form [x1, y1, x2, y2] representing the section   of theworkspace viewable on the sending client.

10) he History Event

History events are pieces of data that are persisted to the database.Any information that is necessary for recreating a visual workspaceshould be sent to the collaborative service via the messages.

Examples

-   -   Creation of notes, images, and other widgets    -   Moving widgets    -   Setting or updating attributes of widgets (e.g. note text,        marker locations)    -   Deleting widgets

When the server receives a history event it does the following:

-   -   Assign the event a unique id    -   Persist the event to the database    -   Broadcast the event, with its id, to all clients connected to        the workspace

History Event Basic Message Format

// server <-- client [client-id, ″he″, target-id, event-type,event-properties]  • client-id (string) the ID of the originating client • target-id (string) the ID of the target object/widget/app to whichthis event is   relevant  • event-type (string) an arbitrary event type • properties (object) a JSON object describing pertinent key / valuesfor the event    ○ regionId (string)  the canvas region identifier ifthe object is created in a     canvas region (optional, will be includedif it was included in the history     event sent by the client)

All properties included in a message will be stored on the server andechoed back to clients. They will also be included in the history sentover http.

// server --> client [client-id, ″he″, target-id, event-id, event-type,event-properties]  • client-id (string) the ID of the originating client • target-id (string) the ID of the target window to which this event isrelevant  • event-id (string) the ID of the event in the database  •event-type (string) an arbitrary event type  • properties (object) aJSON object describing pertinent key / values for the event   ○ regionId(string)  the canvas region identifier if the object is created in a   canvas region (optional, will be included if it was included in thehistory    event sent by the client)

Batch History Events

In order to ensure ordering of tightly coupled events, many can be sentin a batch message by changing the event payload to be an array of eventpayloads.

  // server <-- client [client-id, ″bhe″, [event1, event2, event3,event4]]

In this case, each event is a packet send as a standard web sockethistory message.

The event structure is:

  [targetId, eventType, props] ///

So, the clientId portion is not repeated, but all else is as a standardevent.

Current History Event Types

create Add a widget to the workspace delete Remove the widget from theworkspace position Update the size or location of the widget in theworkspace template Change a card template (background color) membershipUpdate the target children. Used for groups. pin Pin or unpin a widgetstroke Add a pen or eraser stroke to a widget text Sets or update thetext and/or text formatting of a note. markercreate Creates a locationmarker markermove Moves an existing location marker markerdelete Deletesan existing location marker tsxappevent Used for creating, deleting, andupdating tsx widgets such as web browsers navigate Used for navigatingto different page inside group documents (MS docs/PDF)

Widgets and History Events Table

work- web location note image space browser marker pdf group doc createX X * X X X delete X X * X X X position X X * X X X template Xmembership X pin X X X X stroke X X X text X markercreate X markermove Xmarkerdelete X tsxappevent X navigate X

*The browser client supports receiving alternative versions of thesemessages but does not send them out to other clients

History Event Details Comments

Comments are stored in the history database but are associated with aparticular object rather than a position on the plane.

// client --> server [client-id, ″he″, target-id, ″create″, { ″id″:″5123e7ebcd18d3ef5e000001″,  ″type″:″comment″,  ″text″:″text ofthe comment″,  ″parent″:″5123e7ebcd18d3ef5e000000″}] Server will append′name′ to the body of the comment into the props object. The parent  prop is optional and is an id [client-id, ″he″, comment-id, ″delete″}][client-id, ″he″, comment-id, ″text″, {″text″:″text of the comment″}]create

Clients send ‘create’ to the collaboration server to add a widget to aworkspace. For ‘create’ messages the target-id is the id of thecontaining element, usually the workspace-id.

Generic Widget Create Example

// client --> server [client-id, ″he″, workspace-id, ″create″, { ″id″:″5123e7ebcd18d3ef5e000001″,  ″type″:″widget″,  ″regionId″:null }]  • Props    ○ id (string) unique identifier for the widget    ○ type(string) the type of widget    ○ regionId (string) the canvas region ifthe object is created in a canvas     region

Most widgets will also have a location property, usually a rect andorder, but potentially a point or other representation.

Card Create Example

// client --> server [client-id, ″he″, workspace-id, ″create″, { ″id″:″5123e7ebcd18d3ef5e000001″,  ″baseName″:″sessions/all/Teal″, ″ext″:″JPEG″,  ″rect″:[−1298,−390,−1018,−230],  ″actualWidth″:560, ″actualHeight″:320,  ″order″:4,  ″type″:″note″,  ″regionId″:null, ″hidden″:false,  ″text″:″some text for the note″,  ″styles″: {  ″font-size″ : ″42px″,   ″font-weight″ : ″400″,   ″text-transform″ :″inherit″  } }]    • Props     ○ id (string) unique identifier for thewindow     ○ baseName (string) the background image file name     ○ ext(string) the background image file extension     ○ rect (object) thelocation of the window     ○ actualWidth (int) the background imagewidth in pixels     ○ actualHeight (int) the background image height inpixels     ○ order (int) z order     ○ type (string) ″note″ for objectsthat can have text, ″image″ for      other objects     ○ regionId(string) the canvas region if the object is created in a canvas     region     ○ hidden (boolean) whether the window is currentlyhidden     ○ text (string) the text contained in the note (optional)    ○ styles (object) style for the text contained in the note(optional)

PDF Create Example

// server --> client [client-id,  ″he″,  target-id,  event-id, ″create″,  {″type″:″pdf″,   ″id″:″5307ec25a294d9250bf65fce″,    ″assetPath″:″sessions/objects/s7t6mNHxfpqWxAYqYXLF/5307ec25a294d9250bf6    5fce.pdf″,   ″rect″:[1770,284,2994,1076],   ″actualWidth″: 1224,  ″actualHeight″: 792,   ″filename″:″5617_FSPLT1_018078.pdf″,  ″title″:″Record of Decision″,   ″hidden″:false,   ″pin″:false  ″pages″:73}]    • Props      ○ type (string)  ″pdf″      ○ id (string) unique identifier for the pdf      ○ assetPath (string)  the locationof this asset on the asset server. Use       configuration service toget the asset base path.      ○ rect (object) the location of the windowin the workspace      ○ actualWidth (int)  the width of the widest pagein the pdf, combined       with actualHeight to build ″bounding box″     ○ actualHeight (int)  the height of the tallest page in the pdf,combined       with actualWidth to build ″bounding box″      ○ filename(string)  the original file name of the pdf      ○ order (int)  z order     ○ hidden (boolean)  whether the window is currently hidden      ○pin (boolean)  whether the pdf is pinned in one location on the      workspace      ○ regionId (string)  the canvas region if theobject is created in a canvas       region (optional)

Group Create Example

  // client ″-> server [client″id, ″he″ , target-id, ″create″ , {″type″: ″group″ , −id″ : ″S3aS2b392S8f62fce″ , −children″ : [ ]}]  ● • Props  ○ type (string) ″group″   ○ id (strlng) unique identifier for thegroup   ○ children (array) array of target-ld′s of widgets that shouldbe part of the    group

Generic Group Position Example

//client --> server [client-id, he, groupid, ′position′, {″ rect″ : [0,0 , 0 , 0 ], ″order″ :4}]  ● • Props   ○ rect (object) The rect of thegroup. Specified as x1, y1, x2, y2,.   ○ order (int) the z-order of thetarget group

Membership

Replaces the target object's children. Used for grouping items.

// server <-- client [client-id, ″he″, target-id, ″membership″,{″children″ : [53a52b39250f62fce,  53a52b39250f62fce]}] // server -->client [client-id, ″he″, target-id, event-id, ″membership″, {″children″: [53a52b39250f62fce,  53a52b39250f62fce]}]  • Properties   ○ children(array) Required. An array containing at least one widget ID to   include in the group. To remove all children, a delete message shouldbe    used instead.

Group Document Create Example

// server --> client [ client-id, ″he″, target-id, // group document idevent-id, ″create″, { ″type″:″document″, ″rect″:[x1,y1,x2,y2]″maxWidth″:123, ″maxHeight″:456, ″layout″:″stacked″, ″title″:″title ofthis document″, ″assetURL″: ″xxx/xxx/xxx.docx″, ″hidden″: true, ″pin″:false, ″activeChildId″: ″id1838094221″, ″children″: [ ″id0398749123″,″id1838094221″, ″id2849239042″, ″id3849012830″]}]  • Properties   ○ type(string) ″groupdocument″   ○ activeChildId (string) active child Id,such as currently displayed page of    PDF/Doc   ○ children (array)array of child(page) object IDs, array order represents the   child(page) order.   ○ layout (string) Define the client layout torender this group document

Presentation Create Example

// client --> server [client-id, ″he″, target-id, ″create″,{″type″:″presentation″, ″id″:″53a52b39250f62fce″, ″children″: [ ]}]  •Props   ○ type (string) ″presentation″   ○ id (string) unique identifierfor the group   ○ children (array) array of target-id′s of widgets thatshould part of the    presentation in order of presentation PresentationCreate Example // server --> client [ client-id, ″he″, target-id, //presentation id event-id, ″create″, { ″type″:″presentation″, ″children″:[ ″id0398749123″, ″id1838094221″, ″id2849239042″, ″id3849012830″]}]  •Props   ○ type (string) ″presentation″   ○ children (array) array ofchild(page) object IDs, array order represents the    child(page) order.delete

Removes a widget from a workspace.

  // server <-- client [client-id, ″he″, target-id, ″delete″,{″hidden″:true}] // server --> client [client-id, ″he″, target-id,event-id, ″delete″, {″hidden″:true}]positionUsed to save the position of a widget after a move, fling, resize, etc

Generic Widget Position Example

// server <-- client [client-id, ″he″, target-id, ″position″,{new-position}] // server --> client [client-id, ″he″, target-id,event-id, ″position″, {new-position}]  • Props   ○ new-position (object)some way to represent the new position of the    object. See the windowexample.

Generic Window Position Example

// server <-- client [client-id, ″he″, target-id, ″position″,{″rect″:[−1298,−390,−1018,−230],″order″:4}] // server --> client[client-id, ″he″, target-id, event-id, ″position″,{″rect″:[−1298,−390,−1018,−230],″order″:4}]  • Props   ○ rect (object)the location of the target window. Specified as x1, y1, x2, y2   ○ order(int) the z-order of the target windowtemplate

Used to change the template for a note. This allows changing thebackground color.

Note Template Example

// server --> client [client-id, ″he″, workspace-id, event-id,″template″, {″baseName″: ″sessions/all/Beige″}]  • Props   ○ baseName(string) the file name of the new background. The file must    bealready on the collaboration server. The list of templates is availablevia    the http-protocol at /card_templates.json

Used to pin a widget and prevent users from moving or resizing thatwidget. Also used to remove an existing pin.

Generic Widget Position Example

  // server --> client [client-id, ″he″, workspace-id, event-id, ″pin″,{″pin″: true}]  • Props   ○ pin (boolean) true is pin, false is un-pinstroke

Used to add a stroke to a widget or to the workspace background.

Generic Stroke Example

// server <-- client [client-id, ″he″, target-id, ″stroke″, {  ″size″:10,  ″brush″: 1,  ″color″: [255, 153, 0, 1],  ″locs″: [850, 616, 844,617],  ″regionId″: 59.1 }] // server --> client [client-id, ″he″,target-id, event-id, ″stroke″, {  ″size″: 10,  ″brush″: 1,  ″color″:[255, 153, 0, 1],  ″locs″: [850, 616, 844, 617],  ″regionId″: 59.1 }]  • Props    ○ size (integer) diameter of the stroke using thecoordinate space of     the containing object. Strokes on the canvas aresized in world space, while     strokes on widgets are sized in theirparent widget space.    ○ brush (integer) the brush type to use whenrendering the stroke. 1 is     the draw brush, while 2 is the erasebrush.    ○ color (numbers) r/g/b/a values for the color of the stroke.Ignored for     erase strokes (although may still be present).    ○ locs(array) stroke locations in the format: [10, 1, 10, 2, 12, 3]     wherecoordinates are paired [x, y, x, y, x, y, ...] in an array. Similar tosize,     locations are in the coordinate space of the containingobject.    ○ regionId (string) the canvas region if the stroke iscreated in a canvas     region (optional).

Rendering note: strokes should be rendered with end caps centered on thefirst and last points of the stroke. The end cap's diameter should beequal to the brush size. Rendering end caps is the responsibility ofeach client.

text

Set the text and style of text on a widget. Both the text attribute andstyle attribute are optional.

Generic Text Example

// server <-- client [client-id, ″he″, target-id, ″text″, {  ″text″ :″abcdef″,  ″styles″ : {″font-size″ : ″42px″,″font-weight″ :″400″,″text-transform″ : ″inherit″} }] // server --> client [client-id,″he″, target-id, event-id, ″text″, {  ″text″ : ″abcdef″,  ″styles″ : {  ″font-size″ : ″42px″,   ″font-weight″ : ″400″,   ″text-transform″ :″inherit″  } }]    • Props     ○ text (string) the text string to showon the widget     ○ styles (hash) the css styles to apply to the textmarkercreate

Creates a location marker (map marker, waypoint) at a specific place inthe workspace

Example

// server <-- client [client-id, ″he″, new-widget-id, ″markercreate″,{ ″creationTime″:1387565966,  ″name″:″my marker″,  ″y″:1828,  ″x″:−875, ″color″:0 }] // server --> client [client-id, ″he″, new-widget-id,event-id, ″markercreate″,{  ″creationTime″:1387565966,  ″name″:″mymarker″,  ″y″:1828,  ″x″:−875,  ″color″:0 }]   • Props    ○ creationTime(int) the creation time (unix time)    ○ name (string) a label for thelocation marker    ○ y (number) the y location of the marker    ○ x(number) the x location of the marker    ○ template (string) the markertemplate name

Alternative Form Accepted by Browser Client

  // server <-- client [client-id, ″he″, session-id, ″create″,{ ″id″:″52b0f86ac55697ad30003b21″  ″type″:″locationMarker″, ″creationTime″:1387565966,  ″name″:″my marker″,  ″y″:1828,  ″x″:−875, ″template″:″red″ }] // server --> client [client-id, ″he″, session-id,event-id, ″create″,{  ″id″:″52b0f86ac55697ad30003b21″ ″type″:″locationMarker″,  ″creationTime″:1387565966,  ″name″:″mymarker″,  ″y″:1828,  ″x″:−875,  ″template″:″red″ }]markermove

Moves an existing location marker (map marker, waypoint) to a new placein the workspace.

Example

  // server <-- client [client-id, ″he″, marker-id, ″markermove″,{ ″y″:1828,  ″x″:−875, }] // server --> client [client-id, ″he″,marker-id, event-id, ″markermove″,{  ″y″:1828,  ″x″:−875, }]   • Props   ○ y (number) the y location of the marker    ○ x (number) the xlocation of the marker

Alternative Form Accepted by Browser Client

  // server <-- client [client-id, ″he″, target-id, ″position″,{ ″y″:1828,  ″x″:−875, }] // server --> client [client-id, ″he″,target-id, event-id, ″position″,{  ″y″:1828,  ″x″:−875, }]markerdelete

Delete an existing location marker.

Example

  // server <-- client [client-id, ″he″, marker-id, ″markerdelete″,{ }]// server --> client [client-id, ″he″, marker-id, event-id,″markerdelete″,{ }] Alternative form accepted by browser client //server <-- client [client-id, ″he″, target-id, ″delete″,{ ″hidden″:true, }] // server --> client [client-id, ″he″, target-id,event-id, ″delete″,{  ″hidden″:true, }]tsxappevent

TSXappevent sends a history event to various widgets on the tsx system.

Example

// server <-- client [client-id, ″he″, target-id, ″tsxappevent″,{ ″payload″:  {   additional-properties  },  ″messageType″:message-type, ″targetTsxAppId″:tsx-app-id }] // server --> client [client-id, ″he″,target-id, event-id, ″tsxappevent″,{  ″payload″:  {  additional-properties  },  ″messageType″:message-type, ″targetTsxAppId″:tsx-app-id }]    • Props     ○ payload (object) theproperties necessary for this tsxappevent     ○ messageType (string) thetype of message

Example of Creating a Web Browser

// server <-- client [client-id, ″he″,new-browser-id,″tsxappevent″,{ ″payload″: {   ″y″:709,   ″x″:1517,   ″worldSpaceWidth″:800,  ″worldSpaceHeight″:600,   ″windowSpaceWidth″:800,  ″windowSpaceHeight″:600,   ″version″:1,  ″url″:″http://www.google.com/″,   ″order″:735880  }, ″messageType″:″createBrowser″,  ″targetTsxAppId″:″webbrowser″ }] //server --> client [client-id,″he″,new-browser-id, event-id,″tsxappevent″, {  ″payload″: {   ″y″:709,   ″x″:1517,  ″worldSpaceWidth″:800,   ″worldSpaceHeight″:600,  ″windowSpaceWidth″:800,   ″windowSpaceHeight″:600,   ″version″:1,  ″url″:″http://www.google.com/″,   ″order″:735880  }, ″messageType″:″createBrowser″,  ″targetTsxAppId″:″webbrowser″ }]    •Props     ○ payload (object) details needed for creating a browser     ▪ x (number)  the x location of the marker      ▪ y (number)  the ylocation of the marker      ▪ worldSpaceWidth (number)  the width inworld space      ▪ worldSpaceHeight (number)  the height in world space     ▪ windowSpaceWidth (number)   the width in window space      ▪windowSpaceHeight (number)   the height in window space      ▪ version(number)   #TODO      ▪ url (number)   the url this browser widget      should point to messageType *(string) ″createBrowser″ for creating      browsers targetTsxAppId *(string) ″webbrowser″ for web browser      widgets

Example of Deleting a Web Browser

  // client --> server [client-id,″he″,target-id,″tsxappevent″,{ ″messageType″:″deleteBrowser″,  ″targetTsxAppId″:″webbrowser″, ″payload″:{″version″:1} }]navigate

Example of navigating to different item in the payload. One could usethis for example for a browser widget navigating to an URL

  [ client-id, ″he″, target-id, //Group/presentation or maybe BrowserURL ID ″navigate″, payload // navigate to this page ]

11) ve Volatile Event

Volatile events are not recorded in the database, so they are good forin-progress streaming events like dragging a card around the screen, andonce the user lifts their finger, a HistoryEvent is used to record itsfinal place.

Volatile Event Basic Message Format

// server <--> client [client-id, ″ve″, target-id, event-type,event-properties]  • client-id (string) the ID of the originating client • target-id (string) the ID of the target window to which this event isrelevant  • event-type (string) an arbitrary event type  • properties(object) a JSON object describing pertinent key / values for the event

Current Volatile Event Types

sb Begins a new stroke. sc Continues a previously started stroke. seEnds a stroke position Used to share a position with other clients thatshould not be persisted. For example, show a window being dragged inreal time. bf Begin Follow: User A begins to follow User B. Used tonotify User A that user B is following. ef End Follow: User B is nolonger following user A. Used to notify user A that user B is no longerfollowing.Volatile events by Widget type

card image Workspace sb X X X sc X X X se X X X position X X bf X ef X

Workspace

sb Starts a stroke. Used to render strokes on one client while they arebeing drawn on another client. sc Continues a previously started strokeby giving another point to include. Used to render strokes while theyare being drawn on another client. se Ends a previously started stroke.bf Begin Follow: User A begins to follow User B. Used to notify User Athat user B is following. ef End Follow: User B is no longer followinguser A. Used to notify user A that user B is no longer following.

Note

position Live updates the position of a note while its being moved byanother user. sb Starts a stroke. Used to render strokes on one clientwhile they are being drawn on another client. sc Continues a previouslystarted stroke by giving another point to include. Used to renderstrokes while they are being drawn on another client. se Ends apreviously started stroke.

Image

position Live updates the position of an image while its being moved byanother user. sb Starts a stroke. Used to render strokes on one clientwhile they are being drawn on another client. sc Continues a previouslystarted stroke by giving another point to include. Used to renderstrokes while they are being drawn on another client. se Ends apreviously started stroke.

Volatile Event Details

The following fields are properties of several volatile events.

stroke-id Stroke-IDs are selected by the client. Currently they are thesender- id composed with an increasing integer, separated by a dot. Thisis to make it unique within the server context among all clients.target-id A stroke may be attached to a specific target (container) inthe workspace. In the case of a stroke belonging to a widget, the targetID field would contain the ID of the widget. Strokes destined for themain canvas in the workspace are designated by having their target ID bethe same as the workspace id.position—ve

Used to broadcast intermediate steps of a window moving around theworkspace.

Generic Position Example

// server <--> client [client-id, ″ve″, target-id, ″position″,{position-info}]  • Props   ○ position-info - information about thewidget's new position Window Position Example // server <--> client[client-id, ″ve″, target-id, ″position″,{″rect″:[−1298,−390,−1018,−230],″order″:4}]  • Props   ○ rect (object)the location of the target window   ○ order (int) the z-order of thetarget windowsb

Used to broadcast the beginning of a stroke to the other clients.

  // server <--> client [client-id, ″ve″, target-id, ″sb″,{  ″brush″:1, ″size″:2,  ″color″:[214,0,17,1],  ″x″:100,  ″y″:300, ″strokeId″:″395523d316e942b496a2c8a6fe5f2cac″ }]   • Props    ○ x,y(int) the starting point of this stroke    ○ strokeId (string) the ID ofthe new strokesc:

Continues the stroke specified by the stroke id.

  // server <--> client [client-id, ″ve″, target-id, ″sc″, {″x″:100,″y″:300,   ″strokeId″:″395523d316e942b496a2c8a6fe5f2cac″}]  • Props    ○x,y (int) the new end-point of the stroke    ○ strokeId (string) the IDof the new strokese:

Ends the stroke specified by stroke-id.

// server <--> client [client-id, ″ve″, target-id, ″se″,{″strokeId″:″395523d316e942b496a2c8a6fe5f2cac″}]  • stroke-id (string)the ID of the continued strokebf:

Begin Follow: User A begins to follow User B. Used to notify User A thatuser B is following. For this global volatile event, the target ID isthe session id. The user being followed will update the UI to indicatethat user B is following.

  // server <--> client [follower-client-id, ″ve″, session-id, ″bf″, {″clientId″:″395523d316e942b496a2c8a6fe5f2cac″}]  • Props   ○ clientId(string) the ID of the client being followedef:

End Follow: User B is no longer following user A. Used to notify user Athat user B is no longer following. For this global volatile event, thetarget ID is the session id. The user being followed will update the UIto indicate that user B is no longer following. If user B leaves thesession, user A will receive a room list message which does not containuser B. User A's room list will then be rebuilt, no longer showing userB as a follower.

  // server <--> client [follower-client-id, ″ve″, session-id, ″ef″, {″clientId″:″395523d316e942b496a2c8a6fe5f2cac″}]  • Props   ○ clientId(string) the ID of the client no longer being followed

Example Interaction: Moving Objects

A good example illustrating some of theHistoryEvent/VolatileEvent-related changes is moving an object. Whilethe object is being moved/resized by dragging, a series of volatileevents (VEs) is sent to the server, and re-broadcast to all clientssubscribed to the workspace:

// client sends the following volatile events during the move //client->server format is: [<clientId>, <messageType>, <targetId>,<eventType>,   <messageProperties>][″511d6d429b4aee0000000003″,″ve″,″511d6f9c9b4aee0000000039″,″position″,{ ″rect″:[−493,73,−2,565],  ″order″:0 }][″511d6d429b4aee0000000003″,″ve″,″511d6f9c9b4aee0000000039″,″position″,{ ″rect″:[−493,73,−2,565],  ″order″:0 }][″511d6d429b4aee0000000003″,″ve″,″511d6f9c9b4aee0000000039″,″position″,{ ″rect″:[−538,91,−47,583],  ″order″:0 }][″511d6d429b4aee0000000003″,″ve″,″511d6f9c9b4aee0000000039″,″position″,{ ″rect″:[−538,91,−47,583].  ″order″:0 }]

Once the user finishes moving the object, the client should send ahistory event is sent to specify the rect and order of the object:

[″511d6d429b4aee0000000003″,″he″,″511d6f9c9b4aee0000000039″,″position″,{ ″rect″:[−492,73,−1,565].  ″order″:0 }]

The server will respond with the newly persisted he record. Note theinclusion of the record's eventId.

// server-> client format of ′he′ is: [<clientId>, <messageType>,<targetId>, <eventId>, // <eventType>, <messageProps>][″511d6d429b4aee0000000003″,″he″,″511d6f9c9b4aee0000000039″,″511d9165c4223300  00000253″,″position″,{  ″rect″:[−492,73,−1,565],  ″order″:0 }] Note:The eventId will also be included in history that is fetched via theHTTP API.

12) disconnect Disconnect

Inform other app instances opened by the same user to close theirconnection and cease reconnect attempts. This is consumed by browserclients in order to prevent the “frantic reconnect” problem seen whentwo tabs are opened with the same workspace.

  // server --> client [−1, ″disconnect″]

13) ls List Streams

Inform a client of the current streams in a list. Triggered by otherevents, similar to a room list.

// server --> client [send-id, ″1s″, [Stream List for Session]]  •sender-id always −1 (indicating the server initiated the message) Streamlist is an array of objects, each of which contains the followingfields:  • sessionId (string) is the id of the workspace containing theconference  • conferenceId (string) the id of the conference session allusers in this workspace   connect to  • clientId (Object ID) the ID ofthe client broadcasting this particular stream  • streamId (string) theID of this particular AV stream

14) bs Begin Stream

Informs the server of a new AV stream starting. The server responds witha List Streams message.

// server <-- client [sender-id, ″bs″, conferenceId, conferenceProvider,streamId, streamType]  • sender-id clientID of the user starting thestream  • conferenceId (string) the id of the conference session allusers in this workspace   connect to  • conferenceProvider (string) thetype of conference, tokbox or twilio for exmple  • streamId (string) theID of this particular AV stream  • stream Type (string) audio, video orscreenshare

15) es End Stream

Informs the server of a new AV stream ending. The server responds with aList Streams message.

// server <-- client [sender-id, ″es″, conferenceId, streamId]  •sender-id clientID of the user starting the stream  • conferenceId(string) the id of the conference session all users in this workspace  connect to  • streamId (string) the ID of this particular AV stream

16) ss Stream State

Informs the server of an existing AV stream changing state. The serverresponds with a List Streams message.

  // server <-- client [sender-id, ″ss″, streamId, streamType]  •sender-id clientID of the user starting the stream  • streamId (string)the ID of this particular AV stream  • stream Type (string) audio, videoor screenshare

17) oid Object ID Reservation

Use this to create a new unique object id that is acceptable forcreating new history events which create an object.

  ′′′javascript // server <-- client [sender-id, ″oid″]Server responds with:

  // server --> client [″−1″, ′oid′, <new-object-id>]

The API described above provides one example message structure. Otherstructures may be utilized as well, as suits a particularimplementation.

As used herein, the “identification” of an item of information does notnecessarily require the direct specification of that item ofinformation. Information can be “identified” in a field by simplyreferring to the actual information through one or more layers ofindirection, or by identifying one or more items of differentinformation which are together sufficient to determine the actual itemof information. In addition, the term “indicate” is used herein to meanthe same as “identify”.

Also, as used herein, a given signal, event or value is “responsive” toa predecessor signal, event or value if the predecessor signal, event orvalue influenced the given signal, event or value. If there is anintervening processing element, step or time period, the given signal,event or value can still be “responsive” to the predecessor signal,event or value. If the intervening processing element or step combinesmore than one signal, event or value, the signal output of theprocessing element or step is considered “responsive” to each of thesignal, event or value inputs. If the given signal, event or value isthe same as the predecessor signal, event or value, this is merely adegenerate case in which the given signal, event or value is stillconsidered to be “responsive” to the predecessor signal, event or value.“Dependency” of a given signal, event or value upon another signal,event or value is defined similarly.

The applicant hereby discloses in isolation each individual featuredescribed herein and any combination of two or more such features, tothe extent that such features or combinations are capable of beingcarried out based on the present specification as a whole in light ofthe common general knowledge of a person skilled in the art,irrespective of whether such features or combinations of features solveany problems disclosed herein, and without limitation to the scope ofthe claims. The applicant indicates that aspects of the technologydisclosed may consist of any such feature or combination of features. Inview of the foregoing description it will be evident to a person skilledin the art that various modifications may be made within the scope ofthe technology disclosed.

The foregoing description of preferred embodiments of the technologydisclosed has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit thetechnology disclosed to the precise forms disclosed. Obviously, manymodifications and variations will be apparent to practitioners skilledin this art. For example, though the displays described herein are oflarge format, small format displays can also be arranged to use multipledrawing regions, though multiple drawing regions are more useful fordisplays that are at least as large as 12 feet in width. In particular,and without limitation, any and all variations described, suggested bythe Background section of this patent application or by the materialincorporated by reference are specifically incorporated by referenceinto the description herein of embodiments of the technology disclosed.In addition, any and all variations described, suggested or incorporatedby reference herein with respect to any one embodiment are also to beconsidered taught with respect to all other embodiments. The embodimentsdescribed herein were chosen and described in order to best explain theprinciples of the technology disclosed and its practical application,thereby enabling others skilled in the art to understand the technologydisclosed for various embodiments and with various modifications as aresuited to the particular use contemplated. It is intended that the scopeof the technology disclosed be defined by the following claims and theirequivalents.

As with all flowcharts herein, it will be appreciated that many of thesteps can be combined, performed in parallel or performed in a differentsequence without affecting the functions achieved. In some cases, as thereader will appreciate, a rearrangement of steps will achieve the sameresults only if certain other changes are made as well. In other cases,as the reader will appreciate, a rearrangement of steps will achieve thesame results only if certain conditions are satisfied. Furthermore, itwill be appreciated that the flow charts herein show only steps that arepertinent to an understanding of the technology disclosed, and it willbe understood that numerous additional steps for accomplishing otherfunctions can be performed before, after and between those shown.

While the technology disclosed is disclosed by reference to thepreferred embodiments and examples detailed above, it is to beunderstood that these examples are intended in an illustrative ratherthan in a limiting sense. It is contemplated that modifications andcombinations will readily occur to those skilled in the art, whichmodifications and combinations will be within the spirit of thetechnology disclosed and the scope of the following claims. It iscontemplated that technologies described herein can be implemented usingcollaboration data structures other that the spatial event map.

1. A method comprising: storing, in a memory, data including a spatialevent map which locates events in a virtual workspace, the spatial eventmap comprising a specification of a location and dimension of a viewportin the virtual workspace; sending at least a portion of the spatialevent map to a first node; sending data to allow the first node todisplay, in a screen space of a display associated with the first node,graphical objects identified by events in the spatial event map thathave locations within a viewport of the first node; receiving, from thefirst node, a notification of a change to the location or dimension ofthe viewport of the first node; sending data to allow the first node todisplay, in the screen space on the display, graphical objectsidentified by events in the spatial event map that have locations withinthe changed viewport of the first node; receiving a viewport changeevent generated in response to the change of the viewport of the firstnode, the viewport change event being stored in the spatial event mapand comprising a specification of a location and dimension of thechanged viewport of the first node; and sending data to allow the firstnode to follow a second node by, in response to identification of aviewport change event in the spatial event map for the second node,changing the location or dimension of the viewport of the first nodeusing changes of a location or dimension included in the identifiedviewport change event, wherein the changing of the location or dimensionof the viewport of the first node defines the dimension of the viewportof the first node displayed in the screen space of the display so that(i) at least a portion of a viewport of the second node falls within thechanged viewport displayed in the screen space of the display and (ii)the viewport of the first node has at least one of a same zoom level, asame center point and a same aspect ratio as the viewport of the secondnode.
 2. The method of claim 1, wherein the second node includes adisplay having a screen space.
 3. The method of claim 1, wherein thescreen space of the display of the first node and the screen space ofthe display of the second node have different dimensions.
 4. The methodof claim 1, wherein the viewport change event comprises a second node'sspecification of dimensions to be displayed on the viewport of the firstnode, and wherein the viewport of the second node and the viewport ofthe first node are viewports of different or overlapping areas of thevirtual workspace.
 5. The method of claim 1, wherein the viewport of thesecond node and the viewport of the first node are viewports havingcompletely different and non-overlapping areas of the virtual workspace.6. The method of claim 1, further comprising: receiving the viewportchange event from the second node; updating the spatial event map toinclude the viewport change event; and transmitting a message to thefirst node including the viewport change event, wherein the viewportchange event is generated in response to the change of the viewport ofthe second node.
 7. A system comprising: a data processor; memorystoring data including a spatial event map which locates events in avirtual workspace, the spatial event map comprising a specification of alocation and dimension of a viewport in the virtual workspace; logic tosend at least a portion of the spatial event map to a first node; logicto send data to the first node to allow the first node to display, in ascreen space on a display, graphical objects identified by events in thespatial event map that have locations within a viewport of the firstnode; logic to receive, from the first node, a notification of a changein the location or dimension of the viewport of the first node; logic tosend data to the first node to allow the first node to display, in thescreen space on the display, graphical objects identified by events inthe spatial event map that have locations within the changed viewport ofthe first node; logic to receive a viewport change event generated inresponse to the change of the viewport of the first node, the viewportchange event being stored in the spatial event map and comprising aspecification of a location and dimension of the changed viewport of thefirst node; and logic to send data to the first node to allow the firstnode to follow a second node by, in response to identification of aviewport change event in the spatial event map for the second node,changing the location or dimension of the viewport of the first nodeusing changes of a location or dimension included in the identifiedviewport change event, wherein the changing of the location or dimensionof the viewport of the first node defines the dimensions of the viewportof the first node displayed in the screen space of the display so that(i) at least a portion of a viewport of the second node falls within thechanged viewport displayed in the screen space of the display and (ii)the viewport of the first node has at least one of a same zoom level, asame center point and a same aspect ratio as the viewport of the secondnode.
 8. The system of claim 7, wherein the second node includes adisplay having a screen space.
 9. The system of claim 7, wherein thescreen space of the display of the first node and the screen space ofthe display of the second node have different dimensions.
 10. The systemof claim 7, wherein the viewport change event comprises a second node'sspecification of dimensions to be displayed on the viewport of the firstnode, and wherein the viewport of the second node and the viewport ofthe first node are viewports of different or overlapping areas of thevirtual workspace.
 11. The system of claim 7, wherein the viewport ofthe second node and the viewport of the first node are viewports havingcompletely different and non-overlapping areas of the virtual workspace.12. The system of claim 7, further comprising: logic to receive theviewport change event from the second node; logic to update the spatialevent map to include the viewport change event; and logic to transmit amessage to the first node including the viewport change event, whereinthe viewport change event is generated in response to the change of theviewport of the second node.
 13. A non-transitory computer-readablerecording medium having computer instructions recorded thereon, thecomputer instructions, when executed on one or more processors, causingthe one or more processors to perform operations including: storing, ina memory, data including a spatial event map which locates events in avirtual workspace, the spatial event map comprising a specification of alocation and dimension of a viewport in the virtual workspace; sendingat least a portion of the spatial event map to a first node; sendingdata to allow the first node to display, in a screen space of a displayassociated with the first node, graphical objects identified by eventsin the spatial event map that have locations within a viewport of thefirst node; receiving, from the first node, a notification of a changeto the location or dimension of the viewport of the first node; sendingdata to allow the first node to display, in the screen space on thedisplay, graphical objects identified by events in the spatial event mapthat have locations within the changed viewport of the first node;receiving a viewport change event generated in response to the change ofthe viewport of the first node, the viewport change event being storedin the spatial event map and comprising a specification of a locationand dimension of the changed viewport of the first node; and sendingdata to allow the first node to follow a second node by, in response toidentification of a viewport change event in the spatial event map forthe second node, changing the location or dimension of the viewport ofthe first node using changes of a location or dimension included in theidentified viewport change event, wherein the changing of the locationor dimension of the viewport of the first node defines the dimension ofthe viewport of the first node displayed in the screen space of thedisplay so that (i) at least a portion of a viewport of the second nodefalls within the changed viewport displayed in the screen space of thedisplay and (ii) the viewport of the first node has at least one of asame zoom level, a same center point and a same aspect ratio as theviewport of the second node.
 14. The non-transitory computer-readablerecording medium of claim 13, wherein the second node includes a displayhaving a screen space.
 15. The non-transitory computer-readablerecording medium of claim 14, wherein the screen space of the display ofthe first node and the screen space of the display of the second nodehave different dimensions.
 16. The non-transitory computer-readablerecording medium of claim 13, wherein the viewport change eventcomprises a second node's specification of dimensions to be displayed onthe viewport of the first node, and wherein the viewport of the secondnode and the viewport of the first node are viewports of different oroverlapping areas of the virtual workspace.
 17. The non-transitorycomputer-readable recording medium of claim 13, wherein the viewport ofthe second node and the viewport of the first node are viewports havingcompletely different and non-overlapping areas of the virtual workspace.18. The non-transitory computer-readable recording medium of claim 13,the operations further comprising: receiving the viewport change eventfrom the second node; updating the spatial event map to include theviewport change event; and transmitting a message to the first nodeincluding the viewport change event, wherein the viewport change eventis generated in response to the change of the viewport of the secondnode.